From chofreither at gmail.com  Sun Apr  2 12:42:57 2017
From: chofreither at gmail.com (Clemens Hofreither)
Date: Sun, 2 Apr 2017 18:42:57 +0200
Subject: [Cython] Treatment of right-hand side after assignment
Message-ID: <CAJxjEncJPGAJamCY4Q-q00qtTnmCu3-W2MFeJUW=23ESYeJyRA@mail.gmail.com>

Hey all,

(I hope this list isn't dead.)

I'm reading up on the Cython codebase because I'm trying to implement
a small feature (more on that later). There's one thing that really
confuses me currently, and that's the distinction between
generate_post_assignment_code and generate_disposal_code. From the
comments, it appears that exactly one of these should be called, and
which one it is depends on whether a reference from the rhs was
absorbed into the lhs.

So much for the theory, but in practice it seems to work differently.
I'm looking at NameNode.generate_assignment_code() and
IndexNode.generate_assignment_code(), which should do similar things
(generate code for an assignment) except that the lhs has different
structure.

Yet they treat the cleanup of the rhs completely differently.
IndexNode always calls rhs.generate_disposal_code. NameNode almost
always calls rhs.generate_post_assignment_code.

Can anyone shed light on why this cleanup is handled differently
depending on the target of the assignment?

Thanks,
Clemens

From robertwb at gmail.com  Sun Apr  9 02:00:23 2017
From: robertwb at gmail.com (Robert Bradshaw)
Date: Sat, 8 Apr 2017 23:00:23 -0700
Subject: [Cython] Treatment of right-hand side after assignment
In-Reply-To: <CAJxjEncJPGAJamCY4Q-q00qtTnmCu3-W2MFeJUW=23ESYeJyRA@mail.gmail.com>
References: <CAJxjEncJPGAJamCY4Q-q00qtTnmCu3-W2MFeJUW=23ESYeJyRA@mail.gmail.com>
Message-ID: <CADiQ+QBJOXVzmPFkaQbpzJuEAvPyB-hrikE=K9RPgTPkqs0N4A@mail.gmail.com>

On Sun, Apr 2, 2017 at 9:42 AM, Clemens Hofreither <chofreither at gmail.com>
wrote:

> Hey all,
>
> (I hope this list isn't dead.)
>
> I'm reading up on the Cython codebase because I'm trying to implement
> a small feature (more on that later). There's one thing that really
> confuses me currently, and that's the distinction between
> generate_post_assignment_code and generate_disposal_code. From the
> comments, it appears that exactly one of these should be called, and
> which one it is depends on whether a reference from the rhs was
> absorbed into the lhs.
>
> So much for the theory, but in practice it seems to work differently.
> I'm looking at NameNode.generate_assignment_code() and
> IndexNode.generate_assignment_code(), which should do similar things
> (generate code for an assignment) except that the lhs has different
> structure.
>
> Yet they treat the cleanup of the rhs completely differently.
> IndexNode always calls rhs.generate_disposal_code. NameNode almost
> always calls rhs.generate_post_assignment_code.
>
> Can anyone shed light on why this cleanup is handled differently
> depending on the target of the assignment?


It's been a while since I've looked at this, but the best explanation is
that generate_post_assignment_code is used when the rhs is "moved" into the
lhs whereas generate_post_assignment_code is what happens for a "copy."
This is mostly relevant for temps, e.g. when a temp holds a reference to a
Python object, it can be transferred to the assignee (e.g. with NameNodes)
or a new reference can be created for the assignee (e.g. with IndexNodes)
which changes whether the temp lhs needs to be decref'ed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20170408/219addff/attachment.html>

From chofreither at gmail.com  Mon Apr 10 08:57:51 2017
From: chofreither at gmail.com (Clemens Hofreither)
Date: Mon, 10 Apr 2017 14:57:51 +0200
Subject: [Cython] Treatment of right-hand side after assignment
In-Reply-To: <CADiQ+QBJOXVzmPFkaQbpzJuEAvPyB-hrikE=K9RPgTPkqs0N4A@mail.gmail.com>
References: <CAJxjEncJPGAJamCY4Q-q00qtTnmCu3-W2MFeJUW=23ESYeJyRA@mail.gmail.com>
 <CADiQ+QBJOXVzmPFkaQbpzJuEAvPyB-hrikE=K9RPgTPkqs0N4A@mail.gmail.com>
Message-ID: <CAJxjEndu4v-2cYY6CnV6w4MFWpOyg5Go3apGJK_QqpLjLjU4jg@mail.gmail.com>

On Sun, Apr 9, 2017 at 8:00 AM, Robert Bradshaw <robertwb at gmail.com> wrote:
> On Sun, Apr 2, 2017 at 9:42 AM, Clemens Hofreither <chofreither at gmail.com>
> wrote:
>>
>> Hey all,
>>
>> (I hope this list isn't dead.)
>>
>> I'm reading up on the Cython codebase because I'm trying to implement
>> a small feature (more on that later). There's one thing that really
>> confuses me currently, and that's the distinction between
>> generate_post_assignment_code and generate_disposal_code. From the
>> comments, it appears that exactly one of these should be called, and
>> which one it is depends on whether a reference from the rhs was
>> absorbed into the lhs.
>>
>> So much for the theory, but in practice it seems to work differently.
>> I'm looking at NameNode.generate_assignment_code() and
>> IndexNode.generate_assignment_code(), which should do similar things
>> (generate code for an assignment) except that the lhs has different
>> structure.
>>
>> Yet they treat the cleanup of the rhs completely differently.
>> IndexNode always calls rhs.generate_disposal_code. NameNode almost
>> always calls rhs.generate_post_assignment_code.
>>
>> Can anyone shed light on why this cleanup is handled differently
>> depending on the target of the assignment?
>
>
> It's been a while since I've looked at this, but the best explanation is
> that generate_post_assignment_code is used when the rhs is "moved" into the
> lhs whereas generate_post_assignment_code is what happens for a "copy." This
> is mostly relevant for temps, e.g. when a temp holds a reference to a Python
> object, it can be transferred to the assignee (e.g. with NameNodes) or a new
> reference can be created for the assignee (e.g. with IndexNodes) which
> changes whether the temp lhs needs to be decref'ed.
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>

Thanks for your answer. I understood as much about the difference with
temporaries and copies.

My question was why the RIGHT-hand side is treated differently in
NameNode.generate_assignment_code() and
IndexNode.generate_assignment_code(). My understanding is that
NameNode/IndexNode here refers to the type of the left-hand side. But
I don't see why whether or not the RHS is a temporary should depend on
the type of the LHS. Therefore it seems confusing that the code in
these two methods uses different cleanup for the RHS.

Clemens

From robertwb at gmail.com  Mon Apr 10 11:57:06 2017
From: robertwb at gmail.com (Robert Bradshaw)
Date: Mon, 10 Apr 2017 08:57:06 -0700
Subject: [Cython] Treatment of right-hand side after assignment
In-Reply-To: <CAJxjEndu4v-2cYY6CnV6w4MFWpOyg5Go3apGJK_QqpLjLjU4jg@mail.gmail.com>
References: <CAJxjEncJPGAJamCY4Q-q00qtTnmCu3-W2MFeJUW=23ESYeJyRA@mail.gmail.com>
 <CADiQ+QBJOXVzmPFkaQbpzJuEAvPyB-hrikE=K9RPgTPkqs0N4A@mail.gmail.com>
 <CAJxjEndu4v-2cYY6CnV6w4MFWpOyg5Go3apGJK_QqpLjLjU4jg@mail.gmail.com>
Message-ID: <CADiQ+QBeaonEFFwwC0+P3jt5npeUucuPGD0=j7ihg=chFeCWSQ@mail.gmail.com>

On Mon, Apr 10, 2017 at 5:57 AM, Clemens Hofreither
<chofreither at gmail.com> wrote:
>
> On Sun, Apr 9, 2017 at 8:00 AM, Robert Bradshaw <robertwb at gmail.com> wrote:
> > On Sun, Apr 2, 2017 at 9:42 AM, Clemens Hofreither <chofreither at gmail.com>
> > wrote:
> >>
> >> Hey all,
> >>
> >> (I hope this list isn't dead.)
> >>
> >> I'm reading up on the Cython codebase because I'm trying to implement
> >> a small feature (more on that later). There's one thing that really
> >> confuses me currently, and that's the distinction between
> >> generate_post_assignment_code and generate_disposal_code. From the
> >> comments, it appears that exactly one of these should be called, and
> >> which one it is depends on whether a reference from the rhs was
> >> absorbed into the lhs.
> >>
> >> So much for the theory, but in practice it seems to work differently.
> >> I'm looking at NameNode.generate_assignment_code() and
> >> IndexNode.generate_assignment_code(), which should do similar things
> >> (generate code for an assignment) except that the lhs has different
> >> structure.
> >>
> >> Yet they treat the cleanup of the rhs completely differently.
> >> IndexNode always calls rhs.generate_disposal_code. NameNode almost
> >> always calls rhs.generate_post_assignment_code.
> >>
> >> Can anyone shed light on why this cleanup is handled differently
> >> depending on the target of the assignment?
> >
> >
> > It's been a while since I've looked at this, but the best explanation is
> > that generate_post_assignment_code is used when the rhs is "moved" into the
> > lhs whereas generate_post_assignment_code is what happens for a "copy." This
> > is mostly relevant for temps, e.g. when a temp holds a reference to a Python
> > object, it can be transferred to the assignee (e.g. with NameNodes) or a new
> > reference can be created for the assignee (e.g. with IndexNodes) which
> > changes whether the temp lhs needs to be decref'ed.
> >
> > _______________________________________________
> > cython-devel mailing list
> > cython-devel at python.org
> > https://mail.python.org/mailman/listinfo/cython-devel
> >
>
> Thanks for your answer. I understood as much about the difference with
> temporaries and copies.
>
> My question was why the RIGHT-hand side is treated differently in
> NameNode.generate_assignment_code() and
> IndexNode.generate_assignment_code(). My understanding is that
> NameNode/IndexNode here refers to the type of the left-hand side. But
> I don't see why whether or not the RHS is a temporary should depend on
> the type of the LHS. Therefore it seems confusing that the code in
> these two methods uses different cleanup for the RHS.


One can generally "transfer" ownership to a NameNode, but this is not
the case with an IndexNode (as it depends on what the [] operator was
overloaded to do). One could always incref the NameNode and use
generate_disposal_code, but using generate_post_assignment_code saves
an unnecessary incref/decref pair.

- Robert

From jdemeyer at cage.ugent.be  Tue Apr 11 06:00:05 2017
From: jdemeyer at cage.ugent.be (Jeroen Demeyer)
Date: Tue, 11 Apr 2017 12:00:05 +0200
Subject: [Cython] Automatically add include_dirs for externs (#1654)
Message-ID: <58ECA925.5010102@cage.ugent.be>

Dear Cython developers,

as part of a big effort to make SageMath less monolithic and make it 
part of a larger ecosystem, we are splitting off certain modules from 
SageMath as separate packages. cysignals was an early example of this.

Given that we use Cython a lot, it is really important for us that 
inter-package C and Cython interfaces work well. One blocking issue is 
https://github.com/cython/cython/pull/1654

That PR improves the way how "cdef extern from" files are found: it will 
automatically add a disutils include_dirs argument to the Extension if 
it's needed to find to extern file. It is needed whenever you have a 
package which installs .pxd and .h files in site-packages.


Cheers,
Jeroen.

From robertwb at math.washington.edu  Wed Apr 12 17:11:37 2017
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 12 Apr 2017 14:11:37 -0700
Subject: [Cython] Automatically add include_dirs for externs (#1654)
In-Reply-To: <58ECA925.5010102@cage.ugent.be>
References: <58ECA925.5010102@cage.ugent.be>
Message-ID: <CADiQ+QArvvFEwTEjdv3C2gHkNN73UbGxbd2+-oGmCtEavME6Xg@mail.gmail.com>

I'm in favor of the idea (basically attempting to resolve header files
relative to the cython file including them and transitively passing
these paths to modules cimporting them); haven't had time to look at
the code yet or think if there could be negative implications.

On Tue, Apr 11, 2017 at 3:00 AM, Jeroen Demeyer <jdemeyer at cage.ugent.be> wrote:
> Dear Cython developers,
>
> as part of a big effort to make SageMath less monolithic and make it part of
> a larger ecosystem, we are splitting off certain modules from SageMath as
> separate packages. cysignals was an early example of this.
>
> Given that we use Cython a lot, it is really important for us that
> inter-package C and Cython interfaces work well. One blocking issue is
> https://github.com/cython/cython/pull/1654
>
> That PR improves the way how "cdef extern from" files are found: it will
> automatically add a disutils include_dirs argument to the Extension if it's
> needed to find to extern file. It is needed whenever you have a package
> which installs .pxd and .h files in site-packages.
>
>
> Cheers,
> Jeroen.
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel