From robertwb at gmail.com  Tue Dec  3 07:30:51 2013
From: robertwb at gmail.com (Robert Bradshaw)
Date: Mon, 2 Dec 2013 22:30:51 -0800
Subject: [Cython] "embedsignature" and "autotestdict" features in Py3.4
In-Reply-To: <5298BAB8.3080909@behnel.de>
References: <5298BAB8.3080909@behnel.de>
Message-ID: <CADiQ+QDM=9Kf0Hi2AkN6DDxkm3jmraj65GSjbcNw9MM6YmGy8w@mail.gmail.com>

On Fri, Nov 29, 2013 at 8:03 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Hi,
>
> recent changes in Py3.4 affect the functionality of the two directives
> "embedsignature" and "autotestdict".
>
> The so-called "argument clinic" extracts any embedded signatures from
> docstrings and moves them into a new property "__text_signature__". This is
> done at runtime, i.e. when either "__doc__" or "__text_signature__" are
> requested.
>
> http://hg.python.org/cpython/file/6a3e09cd96f3/Objects/methodobject.c#l182
>
> I personally consider this a feature (and it works nicely with the
> signatures that Cython embeds), but you may or may not agree. It broke some
> of our own doctests, at least, because the "__doc__" value that we tested
> for was no longer the same.

I'm fine with this new behavior.

> Regarding the "autotestdict", CPython also got smarter and now finds
> doctests embedded in C implemented functions all by itself. This is clearly
> an improvement compared to the previous behaviour. However, this also means
> that it now executes both the doctest of the function itself and its copy
> in the generated "__test__" dict (stored in the module under that name),
> meaning that it executes all function doctests twice. This also broke some
> of Cython's own doctests for us, which modified global state and thus
> failed the second time.
>
> I'm yet unsure if we should try to do something about this. Currently, the
> explicit doctest dict is generated by default, so all code that uses
> doctests in function docstrings suffers from this. We might be able to make
> Cython a bit smarter so that these docstrings are left out of the test dict
> when (C-)compiling in CPython 3.4. Deciding which ones are still needed
> might be a bit tricky, though...

I think it makes sense to exclude these from the __test__ dict at C
compile time--is it the cpdef ones that it can pick up now?

- Robert

From sturla at molden.no  Wed Dec  4 12:55:21 2013
From: sturla at molden.no (Sturla Molden)
Date: Wed, 4 Dec 2013 12:55:21 +0100
Subject: [Cython] BUG: assignment to typed memoryview
In-Reply-To: <5294BBDB.1090609@grinta.net>
References: <5294BBDB.1090609@grinta.net>
Message-ID: <51753A72-7022-4AF5-BBE3-5645E8D61499@molden.no>


On 26 Nov 2013, at 16:18, Daniele Nicolodi <daniele at grinta.net> wrote:

> Hello,
> 
> I believe there is a bug in how assignment to typed memoryviews is
> handled in the compiler.  I think the following code should be valid code:
> 
>  cdef double[::1] a = np.arange(10, dtype=np.double)
>  cdef double[::1] b = np.empty(a.size // 2)
>  b[:] = a[::2]
> 


Then you are wrong. This should not be valid code. You have declared b to be contiguous, a[::2] is a discontiguous slice. Thus a[::2] cannot be assigned to b.



Sturla


From daniele at grinta.net  Wed Dec  4 13:15:19 2013
From: daniele at grinta.net (Daniele Nicolodi)
Date: Wed, 04 Dec 2013 13:15:19 +0100
Subject: [Cython] BUG: assignment to typed memoryview
In-Reply-To: <51753A72-7022-4AF5-BBE3-5645E8D61499@molden.no>
References: <5294BBDB.1090609@grinta.net>
 <51753A72-7022-4AF5-BBE3-5645E8D61499@molden.no>
Message-ID: <529F1CD7.70802@grinta.net>

On 04/12/2013 12:55, Sturla Molden wrote:
> 
> On 26 Nov 2013, at 16:18, Daniele Nicolodi <daniele at grinta.net> wrote:
> 
>> Hello,
>>
>> I believe there is a bug in how assignment to typed memoryviews is
>> handled in the compiler.  I think the following code should be valid code:
>>
>>  cdef double[::1] a = np.arange(10, dtype=np.double)
>>  cdef double[::1] b = np.empty(a.size // 2)
>>  b[:] = a[::2]
>>
> 
> Then you are wrong. This should not be valid code. You have declared
> b to be contiguous, a[::2] is a discontiguous slice. Thus a[::2]
> cannot be assigned to b.

Hello Sturla,

please note that I'm copying the array elements, not assigning to the
memoryview object: there is an important difference between `b[:] =
a[::2]` and  `b = a[::2]`. Removing the IMHO wrong check, the generated
C code correctly copies the array elements from a to b.

Indeed the following code is perfectly fine:

  cdef double[::1] a = np.arange(10, dtype=np.double)
  cdef double[::1] b = np.empty(a.size // 2)
  cdef double[:] t

  c = a[::2]
  b[:] = c

Thank for looking at this.

Cheers,
Daniele


From vinay_sajip at yahoo.co.uk  Mon Dec 16 11:09:51 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 16 Dec 2013 10:09:51 +0000 (UTC)
Subject: [Cython] Seemingly incorrect specialization of templates
Message-ID: <loom.20131216T110615-505@post.gmane.org>

Given this code in Cython/Includes/libcpp/utility.pxd:

cdef extern from "<utility>" namespace "std":
    cdef cppclass pair[T, U]:
        T first
        U second
        # rest omitted

when the following code in Cython/Includes/libcpp/map.pxd is seen:

from utility cimport pair

cdef extern from "<map>" namespace "std":
    cdef cppclass map[T, U]:
        cppclass iterator:
            pair[T, U]& operator*() nogil
            # rest omitted

The line "pair[T, U]& operator*() nogil" causes a specialization
of pair[T, U] to be created, but using placeholders rather than actual
types. This seems wrong - can someone here confirm whether this is
intentional?

This led to a problem where the generated C++ code for a genuine
specialisation instead generated a declaration like "pair<T, U>" instead
of using the actual arguments. Given this code in
Cython/Includes/libcpp/utility.pxd:

cdef extern from "<utility>" namespace "std":
    cdef cppclass pair[T, U]:
        T first
        U second
        # rest omitted

when the following code in Cython/Includes/libcpp/map.pxd is seen:

from utility cimport pair

cdef extern from "<map>" namespace "std":
    cdef cppclass map[T, U]:
        cppclass iterator:
            pair[T, U]& operator*() nogil
            # rest omitted

The line "pair[T, U]& operator*() nogil" causes a specialization
of pair[T, U] to be created, but using placeholders rather than actual
types. This seems wrong - can someone here confirm whether this is
intentional?

This caused a problem where the generated C++ code for a genuine
specialisation instead generated a declaration like "pair<T, U>" instead
of using the actual arguments.Given this code in
Cython/Includes/libcpp/utility.pxd:

cdef extern from "<utility>" namespace "std":
    cdef cppclass pair[T, U]:
        T first
        U second
        # rest omitted

when the following code in Cython/Includes/libcpp/map.pxd is seen:

from utility cimport pair

cdef extern from "<map>" namespace "std":
    cdef cppclass map[T, U]:
        cppclass iterator:
            pair[T, U]& operator*() nogil
            # rest omitted

The line "pair[T, U]& operator*() nogil" causes a specialization
of pair[T, U] to be created, but using placeholders rather than actual
types. This seems wrong - can someone here confirm whether this is
intentional?

This caused a problem where the generated C++ code for a genuine
specialisation instead generated a declaration like "pair<T, U>" instead
of using the actual arguments.

The specialize call occurs in Nodes.py:TemplatedTypeNode.analyse, where the
template_types passed to specialize are actually all instances of
TemplatePlaceholderType.

Regards,

Vinay Sajip
Given this code in Cython/Includes/libcpp/utility.pxd:

cdef extern from "<utility>" namespace "std":
    cdef cppclass pair[T, U]:
        T first
        U second
        # rest omitted

when the following code in Cython/Includes/libcpp/map.pxd is seen:

from utility cimport pair

cdef extern from "<map>" namespace "std":
    cdef cppclass map[T, U]:
        cppclass iterator:
            pair[T, U]& operator*() nogil
            # rest omitted

The line "pair[T, U]& operator*() nogil" causes a specialization
of pair[T, U] to be created, but using placeholders rather than actual
types. This seems wrong - can someone here confirm whether this is
intentional?

This caused a problem where the generated C++ code for a genuine
specialisation instead generated a declaration like "pair<T, U>" instead
of using the actual arguments. This, of course, failed in g++.

The specialize call occurs in Nodes.py:TemplatedTypeNode.analyse, where the
template_types passed to specialize are actually all instances of
TemplatePlaceholderType. It seems like just the base_type could be used.

Regards,

Vinay Sajip


The specialize call occurs in Nodes.py:TemplatedTypeNode.analyse, where the
template_types passed to specialize are actually all instances of
TemplatePlaceholderType.

Regards,

Vinay Sajip


The specialize call occurs in Nodes.py:TemplatedTypeNode.analyse, where the
template_types passed to specialize are actually all instances of
TemplatePlaceholderType.

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Tue Dec 17 10:38:00 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 17 Dec 2013 09:38:00 +0000 (UTC)
Subject: [Cython] Seemingly incorrect specialization of templates
References: <loom.20131216T110615-505@post.gmane.org>
Message-ID: <loom.20131217T103650-817@post.gmane.org>

Sorry, some parts of my post appear to have been duplicated. Not sure how that 
happened.

Regards,

Vinay Sajip