Automatic conversion with fixed-size C arrays
Hi devs, Being able to convert between simple C structs and Python dictionaries is great, but it doesn't work if there is a fixed-size array field in the struct. It seems like the following struct should be convertible safely: cdef struct state_t: int i, j, k float x[3] float v[3] Along with that, I'd expect that converting between fixed-sized C arrays and Python iterables should work as well: def auto_convert(list ll): cdef int arr[10] = ll # ... return arr If this is deemed something worth doing and if someone is willing to give pointers when I get stuck, I'm offering to put in the development time to implement this. Thoughts? Kurt
On Wed, Jul 16, 2014 at 10:01 AM, Kurt Smith <kwmsmith@gmail.com> wrote:
Hi devs,
Being able to convert between simple C structs and Python dictionaries is great, but it doesn't work if there is a fixed-size array field in the struct. It seems like the following struct should be convertible safely:
cdef struct state_t: int i, j, k float x[3] float v[3]
Along with that, I'd expect that converting between fixed-sized C arrays and Python iterables should work as well:
def auto_convert(list ll): cdef int arr[10] = ll # ... return arr
If this is deemed something worth doing and if someone is willing to give pointers when I get stuck, I'm offering to put in the development time to implement this.
Thoughts?
Yes, this'd be nice to have. One difficulty with arrays is that they can't be returned by value, and so the ordinary from_py_function mechanism (which gets called recursively) would need to be adapted. (Allowing to_py_function to be optinally be called by reference instead of by value could be nice as well from a performance standpoint.) - Robert
On Wed, Jul 16, 2014 at 1:02 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
Yes, this'd be nice to have. One difficulty with arrays is that they can't be returned by value, and so the ordinary from_py_function mechanism (which gets called recursively) would need to be adapted. (Allowing to_py_function to be optinally be called by reference instead of by value could be nice as well from a performance standpoint.)
OK, thanks for the pointers. I'll put some time on this over the weekend. Should I just make a PR when things are ready to review, or should I put up an issue first?
- Robert _______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
On Fri, Jul 18, 2014 at 12:08 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Wed, Jul 16, 2014 at 1:02 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
Yes, this'd be nice to have. One difficulty with arrays is that they can't be returned by value, and so the ordinary from_py_function mechanism (which gets called recursively) would need to be adapted. (Allowing to_py_function to be optinally be called by reference instead of by value could be nice as well from a performance standpoint.)
OK, thanks for the pointers. I'll put some time on this over the weekend. Should I just make a PR when things are ready to review, or should I put up an issue first?
I think this thread is sufficient; looking forward to a pull request. - Robert
On Fri, Jul 18, 2014 at 12:44 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Fri, Jul 18, 2014 at 12:08 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Wed, Jul 16, 2014 at 1:02 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
Yes, this'd be nice to have. One difficulty with arrays is that they can't be returned by value, and so the ordinary from_py_function mechanism (which gets called recursively) would need to be adapted. (Allowing to_py_function to be optinally be called by reference instead of by value could be nice as well from a performance standpoint.)
OK, thanks for the pointers. I'll put some time on this over the weekend. Should I just make a PR when things are ready to review, or should I put up an issue first?
I think this thread is sufficient; looking forward to a pull request.
Don't know if you had time to look at this yet, but another possible hack would be to use the fact that struct { int x[N]; } is bit compatible with int[N] and can be passed/assigned by value. E.g. struct wrapped_array { int x[3]; }; int main(int argc, char** argv) { int src[3]; int dest[3]; *((struct wrapped_array *) &dest) = *((struct wrapped_array *) &src); } OTOH, it might make sense to simply add array members as a special case in struct conversion. - Robert
On Thu, Jul 24, 2014 at 2:07 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Fri, Jul 18, 2014 at 12:08 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Wed, Jul 16, 2014 at 1:02 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
Yes, this'd be nice to have. One difficulty with arrays is that they can't be returned by value, and so the ordinary from_py_function mechanism (which gets called recursively) would need to be adapted. (Allowing to_py_function to be optinally be called by reference instead of by value could be nice as well from a performance standpoint.)
OK, thanks for the pointers. I'll put some time on this over the weekend. Should I just make a PR when things are ready to review, or should I
On Fri, Jul 18, 2014 at 12:44 PM, Robert Bradshaw <robertwb@gmail.com> wrote: put up
an issue first?
I think this thread is sufficient; looking forward to a pull request.
Don't know if you had time to look at this yet,
Yes, I've put in some time. Initial focus is getting cdef int a[10] = obj working, and then using that for structs. Took a little while to re-orient myself with the codebase. I'm working on getting the `to_py_function` and `from_py_function` infrastructure to take arguments by reference; right now I'm getting something hacked into place, and I'd appreciate your review to point out the right way (or at least a better way) to do it. Will likely have a PR for you by this weekend.
but another possible hack would be to use the fact that struct { int x[N]; } is bit compatible with int[N] and can be passed/assigned by value. E.g.
Yes, this is a good point, although if I can get the pass-by-reference working for fixed-size arrays, ideally we can do the same for structs with array members.
struct wrapped_array { int x[3]; };
int main(int argc, char** argv) { int src[3]; int dest[3]; *((struct wrapped_array *) &dest) = *((struct wrapped_array *) &src); }
OTOH, it might make sense to simply add array members as a special case in struct conversion.
- Robert _______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
On Thu, Jul 24, 2014 at 7:13 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Thu, Jul 24, 2014 at 2:07 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Fri, Jul 18, 2014 at 12:44 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Fri, Jul 18, 2014 at 12:08 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Wed, Jul 16, 2014 at 1:02 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
Yes, this'd be nice to have. One difficulty with arrays is that they can't be returned by value, and so the ordinary from_py_function mechanism (which gets called recursively) would need to be adapted. (Allowing to_py_function to be optinally be called by reference instead of by value could be nice as well from a performance standpoint.)
OK, thanks for the pointers. I'll put some time on this over the weekend. Should I just make a PR when things are ready to review, or should I put up an issue first?
I think this thread is sufficient; looking forward to a pull request.
Don't know if you had time to look at this yet,
Yes, I've put in some time. Initial focus is getting
cdef int a[10] = obj
working, and then using that for structs.
Sounds good.
Took a little while to re-orient myself with the codebase.
I'm working on getting the `to_py_function` and `from_py_function` infrastructure to take arguments by reference; right now I'm getting something hacked into place, and I'd appreciate your review to point out the right way (or at least a better way) to do it.
from_py_function always takes its argument (a PyObject*) by reference. It's used as an rvalue, so might not make sense for arrays.
Will likely have a PR for you by this weekend.
Cool.
but another possible hack would be to use the fact that struct { int x[N]; } is bit compatible with int[N] and can be passed/assigned by value. E.g.
Yes, this is a good point, although if I can get the pass-by-reference working for fixed-size arrays, ideally we can do the same for structs with array members.
Agreed. - Robert
On Thu, Jul 24, 2014 at 11:16 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Thu, Jul 24, 2014 at 7:13 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Thu, Jul 24, 2014 at 2:07 PM, Robert Bradshaw <robertwb@gmail.com>
wrote:
On Fri, Jul 18, 2014 at 12:44 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Fri, Jul 18, 2014 at 12:08 PM, Kurt Smith <kwmsmith@gmail.com>
wrote:
On Wed, Jul 16, 2014 at 1:02 PM, Robert Bradshaw <robertwb@gmail.com
wrote:
Yes, this'd be nice to have. One difficulty with arrays is that they can't be returned by value, and so the ordinary from_py_function mechanism (which gets called recursively) would need to be adapted. (Allowing to_py_function to be optinally be called by reference instead of by value could be nice as well from a performance standpoint.)
OK, thanks for the pointers. I'll put some time on this over the weekend. Should I just make a PR when things are ready to review, or should I put up an issue first?
I think this thread is sufficient; looking forward to a pull request.
Don't know if you had time to look at this yet,
Yes, I've put in some time. Initial focus is getting
cdef int a[10] = obj
working, and then using that for structs.
Sounds good.
Took a little while to re-orient myself with the codebase.
I'm working on getting the `to_py_function` and `from_py_function` infrastructure to take arguments by reference; right now I'm getting something hacked into place, and I'd appreciate your review to point out the right way (or at least a better way) to do it.
from_py_function always takes its argument (a PyObject*) by reference. It's used as an rvalue, so might not make sense for arrays.
My bad, I worded my response poorly -- what I mean is that currently the generated code is something like: __pyx_v_t1 = xxx_from_py_function_xxx(temp_py_obj); ___pyx_v_a = __pyx_v_t1; where __pyx_v_a and __pyx_v_t1 are declared as fixed-size C arrays. This won't work, for reasons pointed out. So I'm trying to generate instead: err_code = xxx_from_py_function_xxx(temp_py_obj, &__pyx_v_a[0], 10); if (err_code == -1) { ... } Where the 10 is the array length. The function would initialize the array internally. The python object is passed by reference as before, and the array is also passed by reference. The array is assigned to just once, minimizing copying. We can return a status code to propagate errors, etc. This seems like the cleanest code to me. Thoughts? That's what I mean about "taking arguments by reference".
Will likely have a PR for you by this weekend.
Cool.
but another possible hack would be to use the fact that struct { int x[N]; } is bit compatible with int[N] and can be passed/assigned by value. E.g.
Yes, this is a good point, although if I can get the pass-by-reference working for fixed-size arrays, ideally we can do the same for structs with array members.
Agreed.
- Robert _______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
On Fri, Jul 25, 2014 at 8:24 AM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Thu, Jul 24, 2014 at 11:16 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Thu, Jul 24, 2014 at 7:13 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Thu, Jul 24, 2014 at 2:07 PM, Robert Bradshaw <robertwb@gmail.com>
On Fri, Jul 18, 2014 at 12:44 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Fri, Jul 18, 2014 at 12:08 PM, Kurt Smith <kwmsmith@gmail.com>
wrote:
On Wed, Jul 16, 2014 at 1:02 PM, Robert Bradshaw < robertwb@gmail.com> wrote: > > > Yes, this'd be nice to have. One difficulty with arrays is that
wrote: they
> can't be returned by value, and so the ordinary from_py_function > mechanism (which gets called recursively) would need to be adapted. > (Allowing to_py_function to be optinally be called by reference > instead of by value could be nice as well from a performance > standpoint.)
OK, thanks for the pointers. I'll put some time on this over the weekend. Should I just make a PR when things are ready to review, or should I put up an issue first?
I think this thread is sufficient; looking forward to a pull request.
Don't know if you had time to look at this yet,
Yes, I've put in some time. Initial focus is getting
cdef int a[10] = obj
working, and then using that for structs.
Sounds good.
Took a little while to re-orient myself with the codebase.
I'm working on getting the `to_py_function` and `from_py_function` infrastructure to take arguments by reference; right now I'm getting something hacked into place, and I'd appreciate your review to point out the right way (or at least a better way) to do it.
from_py_function always takes its argument (a PyObject*) by reference. It's used as an rvalue, so might not make sense for arrays.
My bad, I worded my response poorly -- what I mean is that currently the generated code is something like:
__pyx_v_t1 = xxx_from_py_function_xxx(temp_py_obj);
___pyx_v_a = __pyx_v_t1;
where __pyx_v_a and __pyx_v_t1 are declared as fixed-size C arrays.
This won't work, for reasons pointed out.
So I'm trying to generate instead:
err_code = xxx_from_py_function_xxx(temp_py_obj, &__pyx_v_a[0], 10); if (err_code == -1) { ... }
FWIW think you can just write &__pyx_v_a
Where the 10 is the array length. The function would initialize the array internally. The python object is passed by reference as before, and the array is also passed by reference. The array is assigned to just once, minimizing copying. We can return a status code to propagate errors, etc. This seems like the cleanest code to me. Thoughts?
Yes, that makes sense. One of my concerns was whether xxx_from_py_function_xxx was always assigned to an lvalue, enabling this transformation, but I think with error checking it's always assigned to a temporary, so we're OK here. Are you thinking of transforming all xxx_from_py_function_xxx to be of this form? It could make sense for structs, and make for cleaner error checking, but would there be more overhead for simple types like converting to a long (which are currently macros)? If the length is an argument, would one have to check for whether to pass this at every use of the from_py_function? (Maybe that could be encapsulated away into a method on the type, but I think it's used in quite a few places.) All that being said, it'll be really nice to get this working. - Robert
On Fri, Jul 25, 2014 at 1:16 PM, Robert Bradshaw < robertwb@math.washington.edu> wrote:
On Fri, Jul 25, 2014 at 8:24 AM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Thu, Jul 24, 2014 at 11:16 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Thu, Jul 24, 2014 at 7:13 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Thu, Jul 24, 2014 at 2:07 PM, Robert Bradshaw <robertwb@gmail.com>
On Fri, Jul 18, 2014 at 12:44 PM, Robert Bradshaw <robertwb@gmail.com
wrote:
On Fri, Jul 18, 2014 at 12:08 PM, Kurt Smith <kwmsmith@gmail.com> wrote: > On Wed, Jul 16, 2014 at 1:02 PM, Robert Bradshaw < robertwb@gmail.com> > wrote: >> >> >> Yes, this'd be nice to have. One difficulty with arrays is that
wrote: they
>> can't be returned by value, and so the ordinary from_py_function >> mechanism (which gets called recursively) would need to be adapted. >> (Allowing to_py_function to be optinally be called by reference >> instead of by value could be nice as well from a performance >> standpoint.) > > > OK, thanks for the pointers. I'll put some time on this over the > weekend. > Should I just make a PR when things are ready to review, or should I > put up > an issue first?
I think this thread is sufficient; looking forward to a pull request.
Don't know if you had time to look at this yet,
Yes, I've put in some time. Initial focus is getting
cdef int a[10] = obj
working, and then using that for structs.
Sounds good.
Took a little while to re-orient myself with the codebase.
I'm working on getting the `to_py_function` and `from_py_function` infrastructure to take arguments by reference; right now I'm getting something hacked into place, and I'd appreciate your review to point out the right way (or at least a better way) to do it.
from_py_function always takes its argument (a PyObject*) by reference. It's used as an rvalue, so might not make sense for arrays.
My bad, I worded my response poorly -- what I mean is that currently the generated code is something like:
__pyx_v_t1 = xxx_from_py_function_xxx(temp_py_obj);
___pyx_v_a = __pyx_v_t1;
where __pyx_v_a and __pyx_v_t1 are declared as fixed-size C arrays.
This won't work, for reasons pointed out.
So I'm trying to generate instead:
err_code = xxx_from_py_function_xxx(temp_py_obj, &__pyx_v_a[0], 10); if (err_code == -1) { ... }
FWIW think you can just write &__pyx_v_a
Yes, thanks, this form will also generalize to multidimensional fixed-sized arrays. For that matter, we can just pass in __pyx_v_a, which is equivalent to either &__pyx_v_a or &__pyx_v_a[0] (K&R 5.3).
Where the 10 is the array length. The function would initialize the array internally. The python object is passed by reference as before, and the array is also passed by reference. The array is assigned to just once, minimizing copying. We can return a status code to propagate errors, etc. This seems like the cleanest code to me. Thoughts?
Yes, that makes sense.
One of my concerns was whether xxx_from_py_function_xxx was always assigned to an lvalue, enabling this transformation, but I think with error checking it's always assigned to a temporary, so we're OK here.
Sorry, I didn't quite follow that. The lhs is an lvalue at the C level already, right? And we'd want to use the array variable directly in the xxx_from_py_function_xxx call, not a temporary, since the `__pyx_v_a = __pyx_v_t1` temp assignment won't work. What am I missing here? Are you thinking of transforming all xxx_from_py_function_xxx to be of this
form?
No, I was thinking this would kick in for just fixed-size arrays, structs, and perhaps others as makes sense. Everything else would remain as-is.
It could make sense for structs, and make for cleaner error checking, but would there be more overhead for simple types like converting to a long (which are currently macros)?
Agreed. Simple types should remain as-is.
If the length is an argument, would one have to check for whether to pass this at every use of the from_py_function?
No, my thinking is that this transformation (and passing in the array size) would only apply for fixed-size arrays.
(Maybe that could be encapsulated away into a method on the type, but I think it's used in quite a few places.)
All that being said, it'll be really nice to get this working.
- Robert
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
On Fri, Jul 25, 2014 at 1:30 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Fri, Jul 25, 2014 at 1:16 PM, Robert Bradshaw <robertwb@math.washington.edu> wrote:
On Fri, Jul 25, 2014 at 8:24 AM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Thu, Jul 24, 2014 at 11:16 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Thu, Jul 24, 2014 at 7:13 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Thu, Jul 24, 2014 at 2:07 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Fri, Jul 18, 2014 at 12:44 PM, Robert Bradshaw <robertwb@gmail.com> wrote: > On Fri, Jul 18, 2014 at 12:08 PM, Kurt Smith <kwmsmith@gmail.com> > wrote: >> On Wed, Jul 16, 2014 at 1:02 PM, Robert Bradshaw >> <robertwb@gmail.com> >> wrote: >>> >>> >>> Yes, this'd be nice to have. One difficulty with arrays is that >>> they >>> can't be returned by value, and so the ordinary from_py_function >>> mechanism (which gets called recursively) would need to be >>> adapted. >>> (Allowing to_py_function to be optinally be called by reference >>> instead of by value could be nice as well from a performance >>> standpoint.) >> >> >> OK, thanks for the pointers. I'll put some time on this over the >> weekend. >> Should I just make a PR when things are ready to review, or should >> I >> put up >> an issue first? > > I think this thread is sufficient; looking forward to a pull > request.
Don't know if you had time to look at this yet,
Yes, I've put in some time. Initial focus is getting
cdef int a[10] = obj
working, and then using that for structs.
Sounds good.
Took a little while to re-orient myself with the codebase.
I'm working on getting the `to_py_function` and `from_py_function` infrastructure to take arguments by reference; right now I'm getting something hacked into place, and I'd appreciate your review to point out the right way (or at least a better way) to do it.
from_py_function always takes its argument (a PyObject*) by reference. It's used as an rvalue, so might not make sense for arrays.
My bad, I worded my response poorly -- what I mean is that currently the generated code is something like:
__pyx_v_t1 = xxx_from_py_function_xxx(temp_py_obj);
___pyx_v_a = __pyx_v_t1;
where __pyx_v_a and __pyx_v_t1 are declared as fixed-size C arrays.
This won't work, for reasons pointed out.
So I'm trying to generate instead:
err_code = xxx_from_py_function_xxx(temp_py_obj, &__pyx_v_a[0], 10); if (err_code == -1) { ... }
FWIW think you can just write &__pyx_v_a
Yes, thanks, this form will also generalize to multidimensional fixed-sized arrays. For that matter, we can just pass in __pyx_v_a, which is equivalent to either &__pyx_v_a or &__pyx_v_a[0] (K&R 5.3).
Where the 10 is the array length. The function would initialize the array internally. The python object is passed by reference as before, and the array is also passed by reference. The array is assigned to just once, minimizing copying. We can return a status code to propagate errors, etc. This seems like the cleanest code to me. Thoughts?
Yes, that makes sense.
One of my concerns was whether xxx_from_py_function_xxx was always assigned to an lvalue, enabling this transformation, but I think with error checking it's always assigned to a temporary, so we're OK here.
Sorry, I didn't quite follow that. The lhs is an lvalue at the C level already, right? And we'd want to use the array variable directly in the xxx_from_py_function_xxx call, not a temporary, since the `__pyx_v_a = __pyx_v_t1` temp assignment won't work. What am I missing here?
I was worried there might be a case that the return value of xxx_from_py_function_xxx was not immediately assigned to an lvalue, but I think that's never the case.
Are you thinking of transforming all xxx_from_py_function_xxx to be of this form?
No, I was thinking this would kick in for just fixed-size arrays, structs, and perhaps others as makes sense. Everything else would remain as-is.
It could make sense for structs, and make for cleaner error checking, but would there be more overhead for simple types like converting to a long (which are currently macros)?
Agreed. Simple types should remain as-is.
If the length is an argument, would one have to check for whether to pass this at every use of the from_py_function?
No, my thinking is that this transformation (and passing in the array size) would only apply for fixed-size arrays.
Sounds good. - Robert
On Thu, Jul 24, 2014 at 9:13 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
I'm working on getting the `to_py_function` and `from_py_function` infrastructure to take arguments by reference; right now I'm getting something hacked into place, and I'd appreciate your review to point out the right way (or at least a better way) to do it.
Will likely have a PR for you by this weekend.
Update: all the pieces are coming together. I have sorted out all the code generation pieces and nearly have a minimal example working, and will have something for you to review soon. From what I can tell, Cython has some baked-in assumptions when it comes to assignment statements. Telling Cython that in this case, no, don't assign to something at the C level, instead generate a function call and pass in the lvalue by reference takes some doing. I tried to fit things in with Cython's type system and pipelines, but I have no doubt that it can be improved to do things the right way.
On Sun, Jul 27, 2014 at 9:34 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Thu, Jul 24, 2014 at 9:13 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
I'm working on getting the `to_py_function` and `from_py_function` infrastructure to take arguments by reference; right now I'm getting something hacked into place, and I'd appreciate your review to point out the right way (or at least a better way) to do it.
Will likely have a PR for you by this weekend.
Update: all the pieces are coming together.
Cool.
I have sorted out all the code generation pieces and nearly have a minimal example working, and will have something for you to review soon. From what I can tell, Cython has some baked-in assumptions when it comes to assignment statements. Telling Cython that in this case, no, don't assign to something at the C level, instead generate a function call and pass in the lvalue by reference takes some doing.
Yeah, I'm not too surprised about that.
I tried to fit things in with Cython's type system and pipelines, but I have no doubt that it can be improved to do things the right way.
I'll be happy to take a look. - Robert
On Mon, Jul 28, 2014 at 11:02 AM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Sun, Jul 27, 2014 at 9:34 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
I have sorted out all the code generation pieces and nearly have a minimal example working, and will have something for you to review soon. From what I can tell, Cython has some baked-in assumptions when it comes to assignment statements. Telling Cython that in this case, no, don't assign to something at the C level, instead generate a function call and pass in the lvalue by reference takes some doing.
Yeah, I'm not too surprised about that.
I have an initial version working for assigning a Python object to a fixed-size array. It's special cased to C ints and pretty messy in parts, but it gets the job done. I have some ideas on how to generalize the type. Like I said, I'll need your input on the right way to do some things. Will clean it up and put up a PR as soon as I can. Once this is in place, assigning a dict-like object to a struct can build on it. There are a number of other little things that would be nice, like allowing slice assignment: c_arr[5:13] = obj and initializing a dynamic C array with a Python object, provided the extent is specified: cdef int *arr = <int*>malloc(N * sizeof(int)) arr[:N] = obj
On Mon, Jul 28, 2014 at 11:29 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Mon, Jul 28, 2014 at 11:02 AM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Sun, Jul 27, 2014 at 9:34 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
I have sorted out all the code generation pieces and nearly have a minimal example working, and will have something for you to review soon. From what I can tell, Cython has some baked-in assumptions when it comes to assignment statements. Telling Cython that in this case, no, don't assign to something at the C level, instead generate a function call and pass in the lvalue by reference takes some doing.
Yeah, I'm not too surprised about that.
I have an initial version working for assigning a Python object to a fixed-size array.
https://github.com/cython/cython/pull/308 Do you prefer that we continue discussion on this thread, or on the PR?
We can move to the PR, makes it easier to do line commenting. On Tue, Jul 29, 2014 at 9:54 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Mon, Jul 28, 2014 at 11:29 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Mon, Jul 28, 2014 at 11:02 AM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Sun, Jul 27, 2014 at 9:34 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
I have sorted out all the code generation pieces and nearly have a minimal example working, and will have something for you to review soon. From what I can tell, Cython has some baked-in assumptions when it comes to assignment statements. Telling Cython that in this case, no, don't assign to something at the C level, instead generate a function call and pass in the lvalue by reference takes some doing.
Yeah, I'm not too surprised about that.
I have an initial version working for assigning a Python object to a fixed-size array.
https://github.com/cython/cython/pull/308
Do you prefer that we continue discussion on this thread, or on the PR?
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Apologies for the radio silence: the original project that motivated this work has a deadline, so I've had to set this aside for now. I fully intend to return to this once that's done. On Thu, Jul 31, 2014 at 11:34 PM, Robert Bradshaw <robertwb@gmail.com> wrote:
We can move to the PR, makes it easier to do line commenting.
On Tue, Jul 29, 2014 at 9:54 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Mon, Jul 28, 2014 at 11:29 PM, Kurt Smith <kwmsmith@gmail.com> wrote:
On Mon, Jul 28, 2014 at 11:02 AM, Robert Bradshaw <robertwb@gmail.com> wrote:
On Sun, Jul 27, 2014 at 9:34 PM, Kurt Smith <kwmsmith@gmail.com>
wrote:
I have sorted out all the code generation pieces and nearly have a minimal example working, and will have something for you to review soon. From what I can tell, Cython has some baked-in assumptions when it comes to assignment statements. Telling Cython that in this case, no, don't assign to something at the C level, instead generate a function call and pass in the lvalue by reference takes
some
doing.
Yeah, I'm not too surprised about that.
I have an initial version working for assigning a Python object to a fixed-size array.
https://github.com/cython/cython/pull/308
Do you prefer that we continue discussion on this thread, or on the PR?
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Kurt Smith schrieb am 16.07.2014 um 19:01:
Being able to convert between simple C structs and Python dictionaries is great, but it doesn't work if there is a fixed-size array field in the struct. It seems like the following struct should be convertible safely:
cdef struct state_t: int i, j, k float x[3] float v[3]
Along with that, I'd expect that converting between fixed-sized C arrays and Python iterables should work as well:
def auto_convert(list ll): cdef int arr[10] = ll # ... return arr
I implemented this, it's in master. There were multiple places in the code where array assignments were either blocked or special cased (or not special cased yet), so many that I'm sure I missed some. Please give it a try and report any problems. I also had to dig through some custom type conversion code and convert it from C to Cython to make it support more complex type conversions automatically. What should work now is: - assigning C arrays by value (which came more as a side effect) - copying Python iterables to C arrays on assignment (also recursively) - coercing C arrays to Python lists (and tuples if you cast) Pull requests that add more tests are *very* welcome. See carray_coercion.pyx for stuff that works and e_ass.pyx for some error tests. Stefan
participants (4)
-
Kurt Smith -
Robert Bradshaw -
Robert Bradshaw -
Stefan Behnel