From stefan_ml at behnel.de  Tue Sep  1 21:01:23 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 01 Sep 2015 21:01:23 +0200
Subject: [Cython] Compiler crash in MarkOverflowingArithmetic
In-Reply-To: <20150830192947.GA6187@jwilk.net>
References: <20150830192947.GA6187@jwilk.net>
Message-ID: <55E5F603.1090000@behnel.de>

Jakub Wilk schrieb am 30.08.2015 um 21:29:
> Tested with Cython 0.23.1 on i386. You will probably need a bigger number
> to trigger it on a 64-bit machine.
> 
> $ cython test.pyx
> 
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> DEF X = 0x811C9DC5
> cdef void f():
>    cdef unsigned int x = X
>        ^
> ------------------------------------------------------------
> 
> test.pyx:3:9: Compiler crash in MarkOverflowingArithmetic
> 
> ModuleNode.body = StatListNode(test.pyx:2:5)
> StatListNode.stats[0] = CFuncDefNode(test.pyx:2:5,
>    args = [...]/0,
>    modifiers = [...]/0,
>    visibility = 'private')
> CFuncDefNode.body = StatListNode(test.pyx:3:9)
> StatListNode.stats[0] = SingleAssignmentNode(test.pyx:3:9,
>    first = True)
> 
> Compiler crash traceback from this point on:
>  File "/usr/local/lib/python2.7/dist-packages/Cython/Compiler/Visitor.py",
> line 183, in _visit
>    return handler_method(obj)
>  File
> "/usr/local/lib/python2.7/dist-packages/Cython/Compiler/TypeInference.py",
> line 327, in visit_SingleAssignmentNode
>    self.visit_assignment(node.lhs, node.rhs)
>  File
> "/usr/local/lib/python2.7/dist-packages/Cython/Compiler/TypeInference.py",
> line 321, in visit_assignment
>    and Utils.long_literal(rhs.value)):
>  File "/usr/local/lib/python2.7/dist-packages/Cython/Utils.py", line 318,
> in long_literal
>    value = str_to_number(value)
>  File "/usr/local/lib/python2.7/dist-packages/Cython/Utils.py", line 312,
> in str_to_number
>    value = int(value, 0)
> ValueError: invalid literal for int() with base 0: '2166136261L'

Thanks for the report. This should fix it:

https://github.com/cython/cython/commit/ea6414cd293ba2134c3df5b37b158d8e3db79083

Stefan


From carlosjosepita at gmail.com  Wed Sep  2 01:40:50 2015
From: carlosjosepita at gmail.com (Carlos Pita)
Date: Tue, 1 Sep 2015 20:40:50 -0300
Subject: [Cython] [Bug] Cannot cython.cast to 'object'
Message-ID: <CAELgYhfH3GNh_JJ4mqWyHt7v+a2ACR6A-+wMzy9tf7EzSEj4sA@mail.gmail.com>

Something like:

cython.cast('object', x)

won't work because base_type will be None in ExprNodes.py:9323.

Here's is a simple fix, albeit I'm not sure the best one:

base_type = self.base_type.analyse(env) if self.base_type else py_object_type


Cheers
--
Carlos

From stefan_ml at behnel.de  Wed Sep  2 06:43:53 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 02 Sep 2015 06:43:53 +0200
Subject: [Cython] DEF converts byte strings to unicode
In-Reply-To: <20150830174400.GA8322@jwilk.net>
References: <20150830174400.GA8322@jwilk.net>
Message-ID: <55E67E89.3020106@behnel.de>

Jakub Wilk schrieb am 30.08.2015 um 19:44:
> DEF silently converts byte strings to Unicode strings:
> 
> $ cat testcase.pyx
> DEF ABYTE = b'a'
> print type(ABYTE)
> 
> $ python --version
> Python 2.7.10
> 
> $ python -c 'import pyximport as p; p.install(); import testcase'
> <type 'unicode'>
> 
> Tested with Cython 0.23.1.

Thanks for the short reproducer.

https://github.com/cython/cython/commit/061b132e58812e844ef4e35e6444ba21e0f95289

Stefan


From jwilk at jwilk.net  Wed Sep  2 08:16:52 2015
From: jwilk at jwilk.net (Jakub Wilk)
Date: Wed, 2 Sep 2015 08:16:52 +0200
Subject: [Cython] array->memoryview typecast causes compiler crash in
 AnalyseExpressionsTransform
Message-ID: <20150902061652.GA8530@jwilk.net>

In https://bugs.debian.org/793129 Dan Greene reported that trying to 
cast a pointer to a memoryview object causes a compiler crash:

$ python -c 'import cython; cython.inline("<int[:12]>NULL")'
Compiling /home/jwilk/.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx because it changed.
[1/1] Cythonizing /home/jwilk/.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx

Error compiling Cython file:
------------------------------------------------------------
...



def __invoke():
    <int[:12]>NULL
   ^
------------------------------------------------------------

.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4: Compiler crash in AnalyseExpressionsTransform

ModuleNode.body = StatListNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:4:0)
StatListNode.stats[0] = DefNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:4:0,
    modifiers = [...]/0,
    name = u'__invoke',
    py_wrapper_required = True,
    reqd_kw_flags_cname = '0',
    used = True)
File 'Nodes.py', line 430, in analyse_expressions: StatListNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
    is_terminator = True)
File 'Nodes.py', line 4741, in analyse_expressions: ExprStatNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4)
File 'ExprNodes.py', line 460, in analyse_expressions: CythonArrayNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
    is_temp = True,
    mode = 'c',
    use_managed_ref = True)
File 'ExprNodes.py', line 9567, in analyse_types: CythonArrayNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
    is_temp = True,
    mode = 'c',
    use_managed_ref = True)
File 'ExprNodes.py', line 9582, in get_cython_array_type: CythonArrayNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
    is_temp = True,
    mode = 'c',
    use_managed_ref = True)

Compiler crash traceback from this point on:
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Compiler/ExprNodes.py", line 9582, in get_cython_array_type
    return env.global_scope().context.cython_scope.viewscope.lookup("array").type
AttributeError: 'CythonScope' object has no attribute 'viewscope'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Shadow.py", line 114, in inline
    return cython_inline(f, *args, **kwds)
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Build/Inline.py", line 212, in cython_inline
    build_extension.extensions = cythonize([extension], include_path=cython_include_dirs, quiet=quiet)
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 877, in cythonize
    cythonize_one(*args)
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 997, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: /home/jwilk/.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx

-- 
Jakub Wilk

From nathan12343 at gmail.com  Thu Sep  3 21:29:54 2015
From: nathan12343 at gmail.com (Nathan Goldbaum)
Date: Thu, 3 Sep 2015 14:29:54 -0500
Subject: [Cython] cygdb: no debug files were found
Message-ID: <CAJXewOmYg5gbguMdyfDdmq0YZPApZmVO8gyk8kq0vFZfKYaiBA@mail.gmail.com>

Hi all,

I'm following the directions in the link below to debug some cython code
using cygdb:

http://docs.cython.org/src/userguide/debugging.html

Unfortunately, when I try to actually debug my program using cygdb, I get a
"no debug files were found" error.

It looks like the error is getting raised here:
https://github.com/cython/cython/blob/master/Cython/Debugger/Cygdb.py#L33

So for some reason the "cython_debug" folder isn't being generated at build
time. I tried to follow the directions closely, you can see the notes I put
together during the installation process here:
https://bpaste.net/show/6b11f815d449

Unfortunately the library I'm trying to debug (yt:
bitbucket.org/yt_analysis/yt) uses a somewhat nonstandard build setup based
on numpy distutils, and I suspect that is breaking some of the assumptions
that went into the cygdb instructions in the cython docs.

Does anyone have any idea what went wrong here?

For now I'm going to fall back to regular gdb, and debug the autogenerated
cython sources.

Thanks for your help,

Nathan Goldbaum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150903/92c907ae/attachment.html>

From jdemeyer at cage.ugent.be  Sat Sep  5 11:01:22 2015
From: jdemeyer at cage.ugent.be (Jeroen Demeyer)
Date: Sat, 05 Sep 2015 11:01:22 +0200
Subject: [Cython] Bootstrapping broken in master
Message-ID: <55EAAF62.5030909@cage.ugent.be>

Checking out Cython from master (the 0.23.x branch works fine) and 
running "python setup.py install" gives me

Unable to find pgen, not compiling formal grammar.
Compiling module Cython.Compiler.Scanning ...

ERROR: 'module' object has no attribute 'CCodeConfig'

Extension module compilation failed, looks like Cython cannot run
properly on this system.  To work around this, pass the option
"--no-cython-compile".  This will install a pure Python version of
Cython without compiling its own sources.

Traceback (most recent call last):
   File "setup.py", line 289, in <module>
     compile_cython_modules(cython_profile, cython_compile_more, 
cython_with_refnanny)
   File "setup.py", line 233, in compile_cython_modules
     result = compile(pyx_source_file)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Main.py", line 
677, in compile
     return compile_single(source, options, full_module_name)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Main.py", line 
630, in compile_single
     return run_pipeline(source, options, full_module_name)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Main.py", line 
487, in run_pipeline
     err, enddata = Pipeline.run_pipeline(pipeline, source)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Pipeline.py", 
line 334, in run_pipeline
     data = phase(data)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Pipeline.py", 
line 53, in generate_pyx_code_stage
     module_node.process_implementation(options, result)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/ModuleNode.py", line 
118, in process_implementation
     self.generate_c_code(env, options, result)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/ModuleNode.py", line 
317, in generate_c_code
     c_code_config = Code.CCodeConfig(
AttributeError: 'module' object has no attribute 'CCodeConfig'

From stefan_ml at behnel.de  Sat Sep  5 11:26:13 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 05 Sep 2015 11:26:13 +0200
Subject: [Cython] Bootstrapping broken in master
In-Reply-To: <55EAAF62.5030909@cage.ugent.be>
References: <55EAAF62.5030909@cage.ugent.be>
Message-ID: <55EAB535.3@behnel.de>

Jeroen Demeyer schrieb am 05.09.2015 um 11:01:
> Checking out Cython from master (the 0.23.x branch works fine) and running
> "python setup.py install" gives me
> 
> Unable to find pgen, not compiling formal grammar.
> Compiling module Cython.Compiler.Scanning ...
> 
> ERROR: 'module' object has no attribute 'CCodeConfig'

Did you try a clean checkout? I.e. delete all .so files in the tree and
rebuild? This looks like there's a stale "Code.so" in Cython/Compiler/.

Stefan


From stefan_ml at behnel.de  Sat Sep  5 11:22:47 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 05 Sep 2015 11:22:47 +0200
Subject: [Cython] overflow bug?
In-Reply-To: <CANg26EWe7ZLk4RMYjtZvnOBx8QjYsY1SdA4YJL0aaO-zJB2WUA@mail.gmail.com>
References: <CANg26EWe7ZLk4RMYjtZvnOBx8QjYsY1SdA4YJL0aaO-zJB2WUA@mail.gmail.com>
Message-ID: <55EAB467.5080009@behnel.de>

Hi Mark!

Mark Florisson schrieb am 14.07.2015 um 20:34:
> I think this might be a bug (python 3.3 + cython 0.22.1):
> 
>     def f(term=b"12345"):
> 
>     val = int('987278186585')
>     # The below line does not work, because it treats 1 as a constant integer
>     # in the C code (32 bit on my machine). Using 1L does work however.
>     val -= 1 << (len(term) * 8)
>     return val
> 
> print(f())
> 
> This works in pure-python, but Cython generates '1 <<
> __pyx_t_somevar', which I think treats the '1' as an integer (causing
> it to overflow). Using '1L' works in the Cython code however (but that
> may be just my platform).

Thanks for the report. Yes, the generated C code evaluates the shift
operation in C space as

     (1 << (__pyx_t_2 * 8))

And yes, I consider this a bug. It's not entirely easy to find the right
fix, though. I mean, the obvious thing to do would be to evaluate all
left-shift operations in Python space, but that wouldn't be fast for the
"normal" cases. Not sure if there's a point where to draw a line, though...

Stefan


From jdemeyer at cage.ugent.be  Sat Sep  5 11:53:03 2015
From: jdemeyer at cage.ugent.be (Jeroen Demeyer)
Date: Sat, 05 Sep 2015 11:53:03 +0200
Subject: [Cython] overflow bug?
In-Reply-To: <55EAB467.5080009@behnel.de>
References: <CANg26EWe7ZLk4RMYjtZvnOBx8QjYsY1SdA4YJL0aaO-zJB2WUA@mail.gmail.com>
 <55EAB467.5080009@behnel.de>
Message-ID: <55EABB7F.9090803@cage.ugent.be>

On 2015-09-05 11:22, Stefan Behnel wrote:
> Thanks for the report. Yes, the generated C code evaluates the shift
> operation in C space as
>
>       (1 << (__pyx_t_2 * 8))
>
> And yes, I consider this a bug. It's not entirely easy to find the right
> fix, though. I mean, the obvious thing to do would be to evaluate all
> left-shift operations in Python space, but that wouldn't be fast for the
> "normal" cases. Not sure if there's a point where to draw a line, though...

To me, it's not different than overflow when multiplying C integers. So 
I don't really consider it a bug. It is something which could be dealt 
with by the overflowcheck directive.


From jdemeyer at cage.ugent.be  Sat Sep  5 12:17:59 2015
From: jdemeyer at cage.ugent.be (Jeroen Demeyer)
Date: Sat, 05 Sep 2015 12:17:59 +0200
Subject: [Cython] All DEF constants are now unsigned?
Message-ID: <55EAC157.9000105@cage.ugent.be>

Hello Cython-devel,

I noticed that now unsigned constants like

DEF foo = 1

become 1U in the C source code.

This change (i.e. commit ea6414cd293ba2134c3df5b37b158d8e3db79083) broke 
Sage. I haven't managed yet to actually isolate why this change broke 
something.

Was this change of 1 to 1U intentional? Why was it done?

Jeroen.

From jdemeyer at cage.ugent.be  Sat Sep  5 13:49:15 2015
From: jdemeyer at cage.ugent.be (Jeroen Demeyer)
Date: Sat, 05 Sep 2015 13:49:15 +0200
Subject: [Cython] All DEF constants are now unsigned?
In-Reply-To: <55EAC157.9000105@cage.ugent.be>
References: <55EAC157.9000105@cage.ugent.be>
Message-ID: <55EAD6BB.1000608@cage.ugent.be>

This is a bug:

DEF myconst = 1
def foo():
     cdef long x = -myconst
     return x

This returns 4294967295.

From stefan_ml at behnel.de  Sat Sep  5 16:01:34 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 05 Sep 2015 16:01:34 +0200
Subject: [Cython] All DEF constants are now unsigned?
In-Reply-To: <55EAC157.9000105@cage.ugent.be>
References: <55EAC157.9000105@cage.ugent.be>
Message-ID: <55EAF5BE.9070304@behnel.de>

Jeroen Demeyer schrieb am 05.09.2015 um 12:17:
> I noticed that now unsigned constants like
> 
> DEF foo = 1
> 
> become 1U in the C source code.
> 
> This change (i.e. commit ea6414cd293ba2134c3df5b37b158d8e3db79083) broke
> Sage. I haven't managed yet to actually isolate why this change broke
> something.
> 
> Was this change of 1 to 1U intentional? Why was it done?

Pretty much intentional, yes. I changed it in response to this bug report:

http://thread.gmane.org/gmane.comp.python.cython.devel/15774

The problem with DEF constants is that they are Python values that use
Python semantics in expressions before they eventually get converted back
into literals and inserted into the AST. Specifically, they loose any
literals semantics that they initially had in the source code, including
any information about literal C integer types that contributed to their
value, as in this case. Thus, there isn't really a 'correct' way to write
them out as literals and C might be happy with what we output, or not.

I guess it might be better to undo this part of the change. Unsigned
literals are usually not the right way to do it in C, definitely not for
everything. We could then maybe apply the usual 32/64 bits heuristic to add
at least L/LL suffixes at need.

Stefan


From stefan_ml at behnel.de  Sat Sep  5 17:09:56 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 05 Sep 2015 17:09:56 +0200
Subject: [Cython] All DEF constants are now unsigned?
In-Reply-To: <55EAF5BE.9070304@behnel.de>
References: <55EAC157.9000105@cage.ugent.be> <55EAF5BE.9070304@behnel.de>
Message-ID: <55EB05C4.6040109@behnel.de>

Stefan Behnel schrieb am 05.09.2015 um 16:01:
> Jeroen Demeyer schrieb am 05.09.2015 um 12:17:
>> I noticed that now unsigned constants like
>>
>> DEF foo = 1
>>
>> become 1U in the C source code.
>>
>> This change (i.e. commit ea6414cd293ba2134c3df5b37b158d8e3db79083) broke
>> Sage. I haven't managed yet to actually isolate why this change broke
>> something.
>>
>> Was this change of 1 to 1U intentional? Why was it done?
> 
> Pretty much intentional, yes. I changed it in response to this bug report:
> 
> http://thread.gmane.org/gmane.comp.python.cython.devel/15774
> 
> The problem with DEF constants is that they are Python values that use
> Python semantics in expressions before they eventually get converted back
> into literals and inserted into the AST. Specifically, they loose any
> literals semantics that they initially had in the source code, including
> any information about literal C integer types that contributed to their
> value, as in this case. Thus, there isn't really a 'correct' way to write
> them out as literals and C might be happy with what we output, or not.
> 
> I guess it might be better to undo this part of the change. Unsigned
> literals are usually not the right way to do it in C, definitely not for
> everything. We could then maybe apply the usual 32/64 bits heuristic to add
> at least L/LL suffixes at need.

Does this look better?

https://github.com/cython/cython/commit/2ac9ffbab1e868137bf33bf025eb7a5c7e4707ce

It now appends the 'U' suffix only to literals that are used in an unsigned
context (e.g. assignment to unsigned variable), and additionally appends
L/LL suffixes for integers that might exceed the int/long limits. That's
just guessing, but it should generally improve the current handling of
literals and DEF expressions.

Stefan


From stefan_ml at behnel.de  Sat Sep  5 20:32:13 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 05 Sep 2015 20:32:13 +0200
Subject: [Cython] call for contribution: PEP 498 - Literal String
	Interpolation
Message-ID: <55EB352D.4000101@behnel.de>

Hi!

It looks like PEP 498 (f-strings) is going to be accepted soon. Anyone
interested in implementing it for Cython?

https://www.python.org/dev/peps/pep-0498/

It's certainly a bit of work, but it's also very much Python-level
functionality and most of it can be handled in the parser, so I would count
it as an entry-level project for working on Cython. There is already an
initial implementation for CPython (in C) which should provide an answer to
the tricky details of the functionality itself, as well as tests for it
(eventually).

https://bugs.python.org/issue24965

So, if this feature, and if extending the Cython language and working on a
real compiler sounds tempting to you, please speak up on the cython-devel
mailing list.

Stefan

From jdemeyer at cage.ugent.be  Sun Sep  6 10:54:02 2015
From: jdemeyer at cage.ugent.be (Jeroen Demeyer)
Date: Sun, 06 Sep 2015 10:54:02 +0200
Subject: [Cython] All DEF constants are now unsigned?
In-Reply-To: <55EB05C4.6040109@behnel.de>
References: <55EAC157.9000105@cage.ugent.be> <55EAF5BE.9070304@behnel.de>
 <55EB05C4.6040109@behnel.de>
Message-ID: <55EBFF2A.7050603@cage.ugent.be>

On 2015-09-05 17:09, Stefan Behnel wrote:
> It now appends the 'U' suffix only to literals that are used in an unsigned
> context (e.g. assignment to unsigned variable), and additionally appends
> L/LL suffixes for integers that might exceed the int/long limits.

Why not just keep it simple, append no suffixes and let the C/C++ 
compiler deal with it? No heuristics needed.

More concretely, I would handle this as follows:
* do not use any L or U suffix
* write all DEF constants as hexadecimal in the source code

This way, the C/C++ compiler will determine the integral type. It will 
use the first type in the following list which can represent the number:
- int
- unsigned int
- long
- unsigned long
- long long
- unsigned long long

The reason for the *hexadecimal* constants is that the unsigned types 
are not tried for decimal constants.

From stefan_ml at behnel.de  Sun Sep  6 14:30:37 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 06 Sep 2015 14:30:37 +0200
Subject: [Cython] All DEF constants are now unsigned?
In-Reply-To: <55EBFF2A.7050603@cage.ugent.be>
References: <55EAC157.9000105@cage.ugent.be> <55EAF5BE.9070304@behnel.de>
 <55EB05C4.6040109@behnel.de> <55EBFF2A.7050603@cage.ugent.be>
Message-ID: <55EC31ED.800@behnel.de>

Jeroen Demeyer schrieb am 06.09.2015 um 10:54:
> On 2015-09-05 17:09, Stefan Behnel wrote:
>> It now appends the 'U' suffix only to literals that are used in an unsigned
>> context (e.g. assignment to unsigned variable), and additionally appends
>> L/LL suffixes for integers that might exceed the int/long limits.
> 
> Why not just keep it simple, append no suffixes and let the C/C++ compiler
> deal with it? No heuristics needed.
> 
> More concretely, I would handle this as follows:
> * do not use any L or U suffix
> * write all DEF constants as hexadecimal in the source code
> 
> This way, the C/C++ compiler will determine the integral type.

I previously tried that with decimal literals and got warnings in gcc.
Hadn't tried hex literals.


> It will use
> the first type in the following list which can represent the number:
> - int
> - unsigned int
> - long
> - unsigned long
> - long long
> - unsigned long long
> 
> The reason for the *hexadecimal* constants is that the unsigned types are
> not tried for decimal constants.

That's interesting to know (you were quoting 6.4.4 of the C standard
apparently). C - what a language full of wonders.

https://github.com/cython/cython/commit/37f6b07978d3ca1b41aae4d1dd747ee63e3b9265

Thanks!

Stefan


From jdemeyer at cage.ugent.be  Sun Sep  6 19:54:32 2015
From: jdemeyer at cage.ugent.be (Jeroen Demeyer)
Date: Sun, 06 Sep 2015 19:54:32 +0200
Subject: [Cython] All DEF constants are now unsigned?
In-Reply-To: <55EC31ED.800@behnel.de>
References: <55EAC157.9000105@cage.ugent.be> <55EAF5BE.9070304@behnel.de>
 <55EB05C4.6040109@behnel.de> <55EBFF2A.7050603@cage.ugent.be>
 <55EC31ED.800@behnel.de>
Message-ID: <55EC7DD8.1050703@cage.ugent.be>

Thanks, I just tested Sage with latest master and all worked fine!

From stefan_ml at behnel.de  Sun Sep  6 20:32:53 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 06 Sep 2015 20:32:53 +0200
Subject: [Cython] All DEF constants are now unsigned?
In-Reply-To: <55EC7DD8.1050703@cage.ugent.be>
References: <55EAC157.9000105@cage.ugent.be> <55EAF5BE.9070304@behnel.de>
 <55EB05C4.6040109@behnel.de> <55EBFF2A.7050603@cage.ugent.be>
 <55EC31ED.800@behnel.de> <55EC7DD8.1050703@cage.ugent.be>
Message-ID: <55EC86D5.8060308@behnel.de>

Jeroen Demeyer schrieb am 06.09.2015 um 19:54:
> Thanks, I just tested Sage with latest master and all worked fine!

Thanks for testing.

Stefan


From robertwb at gmail.com  Tue Sep  8 00:53:06 2015
From: robertwb at gmail.com (Robert Bradshaw)
Date: Mon, 7 Sep 2015 15:53:06 -0700
Subject: [Cython] All DEF constants are now unsigned?
In-Reply-To: <55EC31ED.800@behnel.de>
References: <55EAC157.9000105@cage.ugent.be> <55EAF5BE.9070304@behnel.de>
 <55EB05C4.6040109@behnel.de> <55EBFF2A.7050603@cage.ugent.be>
 <55EC31ED.800@behnel.de>
Message-ID: <CADiQ+QBCZOk-WtMXsumqzR9yh=OYr8h3d8L+QPjnjWruCvdz0A@mail.gmail.com>

On Sun, Sep 6, 2015 at 5:30 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Jeroen Demeyer schrieb am 06.09.2015 um 10:54:
>> On 2015-09-05 17:09, Stefan Behnel wrote:
>>> It now appends the 'U' suffix only to literals that are used in an unsigned
>>> context (e.g. assignment to unsigned variable), and additionally appends
>>> L/LL suffixes for integers that might exceed the int/long limits.
>>
>> Why not just keep it simple, append no suffixes and let the C/C++ compiler
>> deal with it? No heuristics needed.
>>
>> More concretely, I would handle this as follows:
>> * do not use any L or U suffix
>> * write all DEF constants as hexadecimal in the source code
>>
>> This way, the C/C++ compiler will determine the integral type.
>
> I previously tried that with decimal literals and got warnings in gcc.
> Hadn't tried hex literals.
>
>
>> It will use
>> the first type in the following list which can represent the number:
>> - int
>> - unsigned int
>> - long
>> - unsigned long
>> - long long
>> - unsigned long long
>>
>> The reason for the *hexadecimal* constants is that the unsigned types are
>> not tried for decimal constants.
>
> That's interesting to know (you were quoting 6.4.4 of the C standard
> apparently). C - what a language full of wonders.

Wow, I didn't know that either... thanks for the pointer.

From robertwb at gmail.com  Tue Sep  8 02:15:29 2015
From: robertwb at gmail.com (Robert Bradshaw)
Date: Mon, 7 Sep 2015 17:15:29 -0700
Subject: [Cython] [cython-users] Integer division
In-Reply-To: <CADiQ+QCF5j3oFp3tB-Y0VuoddDeCUq8jja8e5iM6TbxGT9kiDg@mail.gmail.com>
References: <CAGRr6BFsdoKoZrb6Coq2H-JKcHJ8cnrtQk-ijEjn92--8kkxqg@mail.gmail.com>
 <CADiQ+QCczebFAT==Kuc5yoCN=URdiDOhpZU++KYaBfWQAy7ZDg@mail.gmail.com>
 <CAGRr6BH+2O7dMCHxP9Y1V+U488dY14qceVfYEbx2cASC=ACPbA@mail.gmail.com>
 <CADiQ+QC4buPQC5mpZpwA0TcBTr6SgcSqOZ1E9nNdgbuAPkxCFw@mail.gmail.com>
 <CADiQ+QAZMof88aTsQoJb6KnqpDsKexwDFuanf_w0zS35yrQXLg@mail.gmail.com>
 <CAGRr6BGRnXGmMzS=Gai0SoYOZC8jr+9u1X7kZG57ghn0KLSvhQ@mail.gmail.com>
 <CADiQ+QCF5j3oFp3tB-Y0VuoddDeCUq8jja8e5iM6TbxGT9kiDg@mail.gmail.com>
Message-ID: <CADiQ+QBdcS0pohB8b4v_xeSUKVKSB5HzyDNbOBqPrViab3i5ZA@mail.gmail.com>

Or perhaps we should respect float division even with cdivision... Thoughts?

On Mon, Sep 7, 2015 at 5:13 PM, Robert Bradshaw <robertwb at gmail.com> wrote:
> Oh, that's another issue. I'll clarify this in the documentation.
>
> On Mon, Sep 7, 2015 at 3:59 PM, Antony Lee <antony.lee at berkeley.edu> wrote:
>> Actually I realized that it's another problem; setting cdivision to True (as
>> I did in the directive of the example I gave, because I basically always set
>> this to True) overrides __future__ division and restores the C division
>> semantic that int / int = int.  The documentation actually says that the
>> cdivision directive should only affect the case of operands with different
>> signs and of division by zero, not whether int / int = int.
>> Antony
>>
>> 2015-09-07 15:50 GMT-07:00 Robert Bradshaw <robertwb at gmail.com>:
>>>
>>> On Mon, Sep 7, 2015 at 3:46 PM, Robert Bradshaw <robertwb at gmail.com>
>>> wrote:
>>> > On Mon, Sep 7, 2015 at 3:29 PM, Antony Lee <antony.lee at berkeley.edu>
>>> > wrote:
>>> >> Sorry, I intended the entire discussion to apply to the case of Python3
>>> >> semantics.  Indeed, even with language_level=3 and from __future__
>>> >> import
>>> >> division, foo.decl(3, 3) prints "1, 2, 0, 1" whereas foo.nodecl(3, 3)
>>> >> prints
>>> >> "1.0, 0.5, 2.0, 1.0".
>>> >
>>> > from __future__ import division behaves as expected; I just added some
>>> > even more explicit tests:
>>> >
>>> > https://github.com/cython/cython/commit/e61da2eb8a292bc34285c895aade523b6d353414
>>> >
>>> > I'm surprised language_level=3 doesn't automatically set this directive.
>>>
>>> Are you sure you're setting it correctly?
>>> https://github.com/cython/cython/blob/master/tests/run/cython3.pyx#L30
>>>
>>> >> 2015-09-07 15:07 GMT-07:00 Robert Bradshaw <robertwb at gmail.com>:
>>> >>>
>>> >>> On Mon, Sep 7, 2015 at 11:59 AM, Antony Lee <antony.lee at berkeley.edu>
>>> >>> wrote:
>>> >>> > I would like to suggest that division of integers with "/" prints a
>>> >>> > warning
>>> >>> > at compile time.  In my opinion, "/" should be restricted to float
>>> >>> > division,
>>> >>> > and "//" should be used when rounding semantics are required.
>>> >>> >
>>> >>> > Otherwise, we get weird edge cases such as:
>>> >>> >
>>> >>> > #cython: cdivision=True, infer_types=True
>>> >>> > def decl(int m, int n):
>>> >>> >     for i in range(1, m):
>>> >>> >         for j in range(1, n):
>>> >>> >             print(i / j) # integer division
>>> >>> >
>>> >>> > def nodecl(m, n):
>>> >>> >     for i in range(1, m):
>>> >>> >         for j in range(1, n):
>>> >>> >             print(i / j) # float division
>>> >>>
>>> >>> Slight correction: int or float division depending on whether you're
>>> >>> using Python 2 or Python 3. The runtime type of i/j isn't something we
>>> >>> can delegate to the C compiler.
>>> >>>
>>> >>> > (whose behavior may even change if type inference rules changes).
>>> >>> >
>>> >>> > A more extreme choice would be to have "/" always behave like float
>>> >>> > (double)
>>> >>> > division even with integer arguments.
>>> >>>
>>> >>> You mean like putting from __future__ import division at the top of
>>> >>> your file? Or compiling with -3 (for Python 3 language semantics)?
>>> >>>
>>> >>> --
>>> >>>
>>> >>> ---
>>> >>> You received this message because you are subscribed to the Google
>>> >>> Groups
>>> >>> "cython-users" group.
>>> >>> To unsubscribe from this group and stop receiving emails from it, send
>>> >>> an
>>> >>> email to cython-users+unsubscribe at googlegroups.com.
>>> >>> For more options, visit https://groups.google.com/d/optout.
>>> >>
>>> >>
>>> >> --
>>> >>
>>> >> ---
>>> >> You received this message because you are subscribed to the Google
>>> >> Groups
>>> >> "cython-users" group.
>>> >> To unsubscribe from this group and stop receiving emails from it, send
>>> >> an
>>> >> email to cython-users+unsubscribe at googlegroups.com.
>>> >> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "cython-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to cython-users+unsubscribe at googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "cython-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cython-users+unsubscribe at googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.

From robertwb at gmail.com  Tue Sep  8 05:54:37 2015
From: robertwb at gmail.com (Robert Bradshaw)
Date: Mon, 7 Sep 2015 20:54:37 -0700
Subject: [Cython] [cython-users] Integer division
In-Reply-To: <CAGRr6BG82RXjVeU7yH_JL8vLiGeP9XmtAUye3sd3j185YQ+4Tg@mail.gmail.com>
References: <CAGRr6BFsdoKoZrb6Coq2H-JKcHJ8cnrtQk-ijEjn92--8kkxqg@mail.gmail.com>
 <CADiQ+QCczebFAT==Kuc5yoCN=URdiDOhpZU++KYaBfWQAy7ZDg@mail.gmail.com>
 <CAGRr6BH+2O7dMCHxP9Y1V+U488dY14qceVfYEbx2cASC=ACPbA@mail.gmail.com>
 <CADiQ+QC4buPQC5mpZpwA0TcBTr6SgcSqOZ1E9nNdgbuAPkxCFw@mail.gmail.com>
 <CADiQ+QAZMof88aTsQoJb6KnqpDsKexwDFuanf_w0zS35yrQXLg@mail.gmail.com>
 <CAGRr6BGRnXGmMzS=Gai0SoYOZC8jr+9u1X7kZG57ghn0KLSvhQ@mail.gmail.com>
 <CADiQ+QCF5j3oFp3tB-Y0VuoddDeCUq8jja8e5iM6TbxGT9kiDg@mail.gmail.com>
 <CADiQ+QBdcS0pohB8b4v_xeSUKVKSB5HzyDNbOBqPrViab3i5ZA@mail.gmail.com>
 <CAGRr6BG82RXjVeU7yH_JL8vLiGeP9XmtAUye3sd3j185YQ+4Tg@mail.gmail.com>
Message-ID: <CADiQ+QCOWjSnkKkHxDGgi_1bP+O94q2tQc_mKhg365VuijF37A@mail.gmail.com>

On Mon, Sep 7, 2015 at 5:42 PM, Antony Lee <antony.lee at berkeley.edu> wrote:
> My 2c (not cc'ed to cython-devel, as I am not registered):
>
> - language_level=3 implies __future__ division.
> - if __future__ division is set: int / int => float (whether variables are
> untyped (= typed at runtime by python) or typed (including by the
> inferencer))
> - if __future__ division is not set: int / int => int, with
> opposite-sign-case and zero-divisor-case handled depending on cdivision
> - in both cases, int // int => int, with opposite-sign-case and
> zero-divisor-case handled depending on cdivision (also for % and divmod).

That's what I'm leaning towards as well, but it is backwards
incompatible. Anyone else have any opinions?

> 2015-09-07 17:15 GMT-07:00 Robert Bradshaw <robertwb at gmail.com>:
>>
>> Or perhaps we should respect float division even with cdivision...
>> Thoughts?
>>
>> On Mon, Sep 7, 2015 at 5:13 PM, Robert Bradshaw <robertwb at gmail.com>
>> wrote:
>> > Oh, that's another issue. I'll clarify this in the documentation.
>> >
>> > On Mon, Sep 7, 2015 at 3:59 PM, Antony Lee <antony.lee at berkeley.edu>
>> > wrote:
>> >> Actually I realized that it's another problem; setting cdivision to
>> >> True (as
>> >> I did in the directive of the example I gave, because I basically
>> >> always set
>> >> this to True) overrides __future__ division and restores the C division
>> >> semantic that int / int = int.  The documentation actually says that
>> >> the
>> >> cdivision directive should only affect the case of operands with
>> >> different
>> >> signs and of division by zero, not whether int / int = int.
>> >> Antony
>> >>
>> >> 2015-09-07 15:50 GMT-07:00 Robert Bradshaw <robertwb at gmail.com>:
>> >>>
>> >>> On Mon, Sep 7, 2015 at 3:46 PM, Robert Bradshaw <robertwb at gmail.com>
>> >>> wrote:
>> >>> > On Mon, Sep 7, 2015 at 3:29 PM, Antony Lee <antony.lee at berkeley.edu>
>> >>> > wrote:
>> >>> >> Sorry, I intended the entire discussion to apply to the case of
>> >>> >> Python3
>> >>> >> semantics.  Indeed, even with language_level=3 and from __future__
>> >>> >> import
>> >>> >> division, foo.decl(3, 3) prints "1, 2, 0, 1" whereas foo.nodecl(3,
>> >>> >> 3)
>> >>> >> prints
>> >>> >> "1.0, 0.5, 2.0, 1.0".
>> >>> >
>> >>> > from __future__ import division behaves as expected; I just added
>> >>> > some
>> >>> > even more explicit tests:
>> >>> >
>> >>> >
>> >>> > https://github.com/cython/cython/commit/e61da2eb8a292bc34285c895aade523b6d353414
>> >>> >
>> >>> > I'm surprised language_level=3 doesn't automatically set this
>> >>> > directive.
>> >>>
>> >>> Are you sure you're setting it correctly?
>> >>> https://github.com/cython/cython/blob/master/tests/run/cython3.pyx#L30
>> >>>
>> >>> >> 2015-09-07 15:07 GMT-07:00 Robert Bradshaw <robertwb at gmail.com>:
>> >>> >>>
>> >>> >>> On Mon, Sep 7, 2015 at 11:59 AM, Antony Lee
>> >>> >>> <antony.lee at berkeley.edu>
>> >>> >>> wrote:
>> >>> >>> > I would like to suggest that division of integers with "/"
>> >>> >>> > prints a
>> >>> >>> > warning
>> >>> >>> > at compile time.  In my opinion, "/" should be restricted to
>> >>> >>> > float
>> >>> >>> > division,
>> >>> >>> > and "//" should be used when rounding semantics are required.
>> >>> >>> >
>> >>> >>> > Otherwise, we get weird edge cases such as:
>> >>> >>> >
>> >>> >>> > #cython: cdivision=True, infer_types=True
>> >>> >>> > def decl(int m, int n):
>> >>> >>> >     for i in range(1, m):
>> >>> >>> >         for j in range(1, n):
>> >>> >>> >             print(i / j) # integer division
>> >>> >>> >
>> >>> >>> > def nodecl(m, n):
>> >>> >>> >     for i in range(1, m):
>> >>> >>> >         for j in range(1, n):
>> >>> >>> >             print(i / j) # float division
>> >>> >>>
>> >>> >>> Slight correction: int or float division depending on whether
>> >>> >>> you're
>> >>> >>> using Python 2 or Python 3. The runtime type of i/j isn't
>> >>> >>> something we
>> >>> >>> can delegate to the C compiler.
>> >>> >>>
>> >>> >>> > (whose behavior may even change if type inference rules
>> >>> >>> > changes).
>> >>> >>> >
>> >>> >>> > A more extreme choice would be to have "/" always behave like
>> >>> >>> > float
>> >>> >>> > (double)
>> >>> >>> > division even with integer arguments.
>> >>> >>>
>> >>> >>> You mean like putting from __future__ import division at the top
>> >>> >>> of
>> >>> >>> your file? Or compiling with -3 (for Python 3 language semantics)?
>> >>> >>>
>> >>> >>> --
>> >>> >>>
>> >>> >>> ---
>> >>> >>> You received this message because you are subscribed to the Google
>> >>> >>> Groups
>> >>> >>> "cython-users" group.
>> >>> >>> To unsubscribe from this group and stop receiving emails from it,
>> >>> >>> send
>> >>> >>> an
>> >>> >>> email to cython-users+unsubscribe at googlegroups.com.
>> >>> >>> For more options, visit https://groups.google.com/d/optout.
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >>
>> >>> >> ---
>> >>> >> You received this message because you are subscribed to the Google
>> >>> >> Groups
>> >>> >> "cython-users" group.
>> >>> >> To unsubscribe from this group and stop receiving emails from it,
>> >>> >> send
>> >>> >> an
>> >>> >> email to cython-users+unsubscribe at googlegroups.com.
>> >>> >> For more options, visit https://groups.google.com/d/optout.
>> >>>
>> >>> --
>> >>>
>> >>> ---
>> >>> You received this message because you are subscribed to the Google
>> >>> Groups
>> >>> "cython-users" group.
>> >>> To unsubscribe from this group and stop receiving emails from it, send
>> >>> an
>> >>> email to cython-users+unsubscribe at googlegroups.com.
>> >>> For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >> --
>> >>
>> >> ---
>> >> You received this message because you are subscribed to the Google
>> >> Groups
>> >> "cython-users" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> >> an
>> >> email to cython-users+unsubscribe at googlegroups.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "cython-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cython-users+unsubscribe at googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

From insertinterestingnamehere at gmail.com  Tue Sep  8 06:54:04 2015
From: insertinterestingnamehere at gmail.com (Ian Henriksen)
Date: Tue, 08 Sep 2015 04:54:04 +0000
Subject: [Cython] [cython-users] Integer division
In-Reply-To: <CADiQ+QCOWjSnkKkHxDGgi_1bP+O94q2tQc_mKhg365VuijF37A@mail.gmail.com>
References: <CAGRr6BFsdoKoZrb6Coq2H-JKcHJ8cnrtQk-ijEjn92--8kkxqg@mail.gmail.com>
 <CADiQ+QCczebFAT==Kuc5yoCN=URdiDOhpZU++KYaBfWQAy7ZDg@mail.gmail.com>
 <CAGRr6BH+2O7dMCHxP9Y1V+U488dY14qceVfYEbx2cASC=ACPbA@mail.gmail.com>
 <CADiQ+QC4buPQC5mpZpwA0TcBTr6SgcSqOZ1E9nNdgbuAPkxCFw@mail.gmail.com>
 <CADiQ+QAZMof88aTsQoJb6KnqpDsKexwDFuanf_w0zS35yrQXLg@mail.gmail.com>
 <CAGRr6BGRnXGmMzS=Gai0SoYOZC8jr+9u1X7kZG57ghn0KLSvhQ@mail.gmail.com>
 <CADiQ+QCF5j3oFp3tB-Y0VuoddDeCUq8jja8e5iM6TbxGT9kiDg@mail.gmail.com>
 <CADiQ+QBdcS0pohB8b4v_xeSUKVKSB5HzyDNbOBqPrViab3i5ZA@mail.gmail.com>
 <CAGRr6BG82RXjVeU7yH_JL8vLiGeP9XmtAUye3sd3j185YQ+4Tg@mail.gmail.com>
 <CADiQ+QCOWjSnkKkHxDGgi_1bP+O94q2tQc_mKhg365VuijF37A@mail.gmail.com>
Message-ID: <CAPWbfpQp6K+o7wqmY2g9xCc_Wy8=BeN8ytpe_63_sdiXpSkPVQ@mail.gmail.com>

Since Cython already tries to make division of C integers behave like
Python
division, I'd expect python-like behavior (mimicking python 2 or 3 as
needed) by
default and unchanged C integer division when the cdivision directive is
set.
Best,
-Ian Henriksen

On Mon, Sep 7, 2015 at 9:55 PM Robert Bradshaw <robertwb at gmail.com> wrote:

> On Mon, Sep 7, 2015 at 5:42 PM, Antony Lee <antony.lee at berkeley.edu>
> wrote:
> > My 2c (not cc'ed to cython-devel, as I am not registered):
> >
> > - language_level=3 implies __future__ division.
> > - if __future__ division is set: int / int => float (whether variables
> are
> > untyped (= typed at runtime by python) or typed (including by the
> > inferencer))
> > - if __future__ division is not set: int / int => int, with
> > opposite-sign-case and zero-divisor-case handled depending on cdivision
> > - in both cases, int // int => int, with opposite-sign-case and
> > zero-divisor-case handled depending on cdivision (also for % and divmod).
>
> That's what I'm leaning towards as well, but it is backwards
> incompatible. Anyone else have any opinions?
>
> > 2015-09-07 17:15 GMT-07:00 Robert Bradshaw <robertwb at gmail.com>:
> >>
> >> Or perhaps we should respect float division even with cdivision...
> >> Thoughts?
> >>
> >> On Mon, Sep 7, 2015 at 5:13 PM, Robert Bradshaw <robertwb at gmail.com>
> >> wrote:
> >> > Oh, that's another issue. I'll clarify this in the documentation.
> >> >
> >> > On Mon, Sep 7, 2015 at 3:59 PM, Antony Lee <antony.lee at berkeley.edu>
> >> > wrote:
> >> >> Actually I realized that it's another problem; setting cdivision to
> >> >> True (as
> >> >> I did in the directive of the example I gave, because I basically
> >> >> always set
> >> >> this to True) overrides __future__ division and restores the C
> division
> >> >> semantic that int / int = int.  The documentation actually says that
> >> >> the
> >> >> cdivision directive should only affect the case of operands with
> >> >> different
> >> >> signs and of division by zero, not whether int / int = int.
> >> >> Antony
> >> >>
> >> >> 2015-09-07 15:50 GMT-07:00 Robert Bradshaw <robertwb at gmail.com>:
> >> >>>
> >> >>> On Mon, Sep 7, 2015 at 3:46 PM, Robert Bradshaw <robertwb at gmail.com
> >
> >> >>> wrote:
> >> >>> > On Mon, Sep 7, 2015 at 3:29 PM, Antony Lee <
> antony.lee at berkeley.edu>
> >> >>> > wrote:
> >> >>> >> Sorry, I intended the entire discussion to apply to the case of
> >> >>> >> Python3
> >> >>> >> semantics.  Indeed, even with language_level=3 and from
> __future__
> >> >>> >> import
> >> >>> >> division, foo.decl(3, 3) prints "1, 2, 0, 1" whereas
> foo.nodecl(3,
> >> >>> >> 3)
> >> >>> >> prints
> >> >>> >> "1.0, 0.5, 2.0, 1.0".
> >> >>> >
> >> >>> > from __future__ import division behaves as expected; I just added
> >> >>> > some
> >> >>> > even more explicit tests:
> >> >>> >
> >> >>> >
> >> >>> >
> https://github.com/cython/cython/commit/e61da2eb8a292bc34285c895aade523b6d353414
> >> >>> >
> >> >>> > I'm surprised language_level=3 doesn't automatically set this
> >> >>> > directive.
> >> >>>
> >> >>> Are you sure you're setting it correctly?
> >> >>>
> https://github.com/cython/cython/blob/master/tests/run/cython3.pyx#L30
> >> >>>
> >> >>> >> 2015-09-07 15:07 GMT-07:00 Robert Bradshaw <robertwb at gmail.com>:
> >> >>> >>>
> >> >>> >>> On Mon, Sep 7, 2015 at 11:59 AM, Antony Lee
> >> >>> >>> <antony.lee at berkeley.edu>
> >> >>> >>> wrote:
> >> >>> >>> > I would like to suggest that division of integers with "/"
> >> >>> >>> > prints a
> >> >>> >>> > warning
> >> >>> >>> > at compile time.  In my opinion, "/" should be restricted to
> >> >>> >>> > float
> >> >>> >>> > division,
> >> >>> >>> > and "//" should be used when rounding semantics are required.
> >> >>> >>> >
> >> >>> >>> > Otherwise, we get weird edge cases such as:
> >> >>> >>> >
> >> >>> >>> > #cython: cdivision=True, infer_types=True
> >> >>> >>> > def decl(int m, int n):
> >> >>> >>> >     for i in range(1, m):
> >> >>> >>> >         for j in range(1, n):
> >> >>> >>> >             print(i / j) # integer division
> >> >>> >>> >
> >> >>> >>> > def nodecl(m, n):
> >> >>> >>> >     for i in range(1, m):
> >> >>> >>> >         for j in range(1, n):
> >> >>> >>> >             print(i / j) # float division
> >> >>> >>>
> >> >>> >>> Slight correction: int or float division depending on whether
> >> >>> >>> you're
> >> >>> >>> using Python 2 or Python 3. The runtime type of i/j isn't
> >> >>> >>> something we
> >> >>> >>> can delegate to the C compiler.
> >> >>> >>>
> >> >>> >>> > (whose behavior may even change if type inference rules
> >> >>> >>> > changes).
> >> >>> >>> >
> >> >>> >>> > A more extreme choice would be to have "/" always behave like
> >> >>> >>> > float
> >> >>> >>> > (double)
> >> >>> >>> > division even with integer arguments.
> >> >>> >>>
> >> >>> >>> You mean like putting from __future__ import division at the top
> >> >>> >>> of
> >> >>> >>> your file? Or compiling with -3 (for Python 3 language
> semantics)?
> >> >>> >>>
> >> >>> >>> --
> >> >>> >>>
> >> >>> >>> ---
> >> >>> >>> You received this message because you are subscribed to the
> Google
> >> >>> >>> Groups
> >> >>> >>> "cython-users" group.
> >> >>> >>> To unsubscribe from this group and stop receiving emails from
> it,
> >> >>> >>> send
> >> >>> >>> an
> >> >>> >>> email to cython-users+unsubscribe at googlegroups.com.
> >> >>> >>> For more options, visit https://groups.google.com/d/optout.
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >>
> >> >>> >> ---
> >> >>> >> You received this message because you are subscribed to the
> Google
> >> >>> >> Groups
> >> >>> >> "cython-users" group.
> >> >>> >> To unsubscribe from this group and stop receiving emails from it,
> >> >>> >> send
> >> >>> >> an
> >> >>> >> email to cython-users+unsubscribe at googlegroups.com.
> >> >>> >> For more options, visit https://groups.google.com/d/optout.
> >> >>>
> >> >>> --
> >> >>>
> >> >>> ---
> >> >>> You received this message because you are subscribed to the Google
> >> >>> Groups
> >> >>> "cython-users" group.
> >> >>> To unsubscribe from this group and stop receiving emails from it,
> send
> >> >>> an
> >> >>> email to cython-users+unsubscribe at googlegroups.com.
> >> >>> For more options, visit https://groups.google.com/d/optout.
> >> >>
> >> >>
> >> >> --
> >> >>
> >> >> ---
> >> >> You received this message because you are subscribed to the Google
> >> >> Groups
> >> >> "cython-users" group.
> >> >> To unsubscribe from this group and stop receiving emails from it,
> send
> >> >> an
> >> >> email to cython-users+unsubscribe at googlegroups.com.
> >> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >> --
> >>
> >> ---
> >> You received this message because you are subscribed to the Google
> Groups
> >> "cython-users" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an
> >> email to cython-users+unsubscribe at googlegroups.com.
> >> For more options, visit https://groups.google.com/d/optout.
> >
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google Groups
> > "cython-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to cython-users+unsubscribe at googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150908/c2ba36ba/attachment-0001.html>

From zauddelig at gmail.com  Mon Sep  7 12:23:34 2015
From: zauddelig at gmail.com (Fabrizio Messina)
Date: Mon, 7 Sep 2015 18:23:34 +0800
Subject: [Cython] call for contribution: PEP 498 - Literal String
	Interpolation
In-Reply-To: <CAJZdaUVSZQotBd-SfCXDKDa0Q-06gcfGOQh5VjXCb3obopaAMA@mail.gmail.com>
References: <55EB352D.4000101@behnel.de>
 <CAJZdaUXWDzbqT8M9iF12Y9NMDnryL5WK5Q4aAkKRY2UvzcogPQ@mail.gmail.com>
 <CAJZdaUVSZQotBd-SfCXDKDa0Q-06gcfGOQh5VjXCb3obopaAMA@mail.gmail.com>
Message-ID: <CAJZdaUVf8kL2bP8m7uyt16OtUTnx5VK8yMQ5WvGTCUgp1Aj3Sg@mail.gmail.com>

Hello, I would like to give a look on that on my spare time.
On 5 Sep 2015 20:32, "Stefan Behnel" <stefan_ml at behnel.de> wrote:

Hi!

It looks like PEP 498 (f-strings) is going to be accepted soon. Anyone
interested in implementing it for Cython?

https://www.python.org/dev/peps/pep-0498/

It's certainly a bit of work, but it's also very much Python-level
functionality and most of it can be handled in the parser, so I would count
it as an entry-level project for working on Cython. There is already an
initial implementation for CPython (in C) which should provide an answer to
the tricky details of the functionality itself, as well as tests for it
(eventually).

https://bugs.python.org/issue24965

So, if this feature, and if extending the Cython language and working on a
real compiler sounds tempting to you, please speak up on the cython-devel
mailing list.

Stefan
_______________________________________________
cython-devel mailing list
cython-devel at python.org
https://mail.python.org/mailman/listinfo/cython-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150907/86461c32/attachment.html>

From stefan_ml at behnel.de  Tue Sep  8 08:18:50 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 08 Sep 2015 08:18:50 +0200
Subject: [Cython] call for contribution: PEP 498 - Literal String
	Interpolation
In-Reply-To: <CAJZdaUVf8kL2bP8m7uyt16OtUTnx5VK8yMQ5WvGTCUgp1Aj3Sg@mail.gmail.com>
References: <55EB352D.4000101@behnel.de>
 <CAJZdaUXWDzbqT8M9iF12Y9NMDnryL5WK5Q4aAkKRY2UvzcogPQ@mail.gmail.com>
 <CAJZdaUVSZQotBd-SfCXDKDa0Q-06gcfGOQh5VjXCb3obopaAMA@mail.gmail.com>
 <CAJZdaUVf8kL2bP8m7uyt16OtUTnx5VK8yMQ5WvGTCUgp1Aj3Sg@mail.gmail.com>
Message-ID: <55EE7DCA.9050108@behnel.de>

Fabrizio Messina schrieb am 07.09.2015 um 12:23:
> On 5 Sep 2015 20:32, "Stefan Behnel" wrote:
>> It looks like PEP 498 (f-strings) is going to be accepted soon. Anyone
>> interested in implementing it for Cython?
>> 
>> https://www.python.org/dev/peps/pep-0498/
>> 
>> It's certainly a bit of work, but it's also very much Python-level
>> functionality and most of it can be handled in the parser, so I would count
>> it as an entry-level project for working on Cython. There is already an
>> initial implementation for CPython (in C) which should provide an answer to
>> the tricky details of the functionality itself, as well as tests for it
>> (eventually).
>> 
>> https://bugs.python.org/issue24965
>> 
>> So, if this feature, and if extending the Cython language and working on a
>> real compiler sounds tempting to you, please speak up on the cython-devel
>> mailing list.
> 
> Hello, I would like to give a look on that on my spare time.

Cool. If you need any help or have any questions, just reply to this thread.

Here's an intro to get you started. It explains how the test suite works
and how you can figure out how the C code you see is being generated.

https://github.com/cython/cython/wiki/HackerGuide#getting-started

Stefan


From zauddelig at gmail.com  Tue Sep  8 13:09:32 2015
From: zauddelig at gmail.com (Fabrizio Messina)
Date: Tue, 8 Sep 2015 19:09:32 +0800
Subject: [Cython] call for contribution: PEP 498 - Literal String
	Interpolation
In-Reply-To: <55EE7DCA.9050108@behnel.de>
References: <55EB352D.4000101@behnel.de>
 <CAJZdaUXWDzbqT8M9iF12Y9NMDnryL5WK5Q4aAkKRY2UvzcogPQ@mail.gmail.com>
 <CAJZdaUVSZQotBd-SfCXDKDa0Q-06gcfGOQh5VjXCb3obopaAMA@mail.gmail.com>
 <CAJZdaUVf8kL2bP8m7uyt16OtUTnx5VK8yMQ5WvGTCUgp1Aj3Sg@mail.gmail.com>
 <55EE7DCA.9050108@behnel.de>
Message-ID: <CAJZdaUVgRCeARcyrcp0+=kjQJxiibCZJx6xwZFAYj03Wa=fvZg@mail.gmail.com>

Thank you I will give it a look this evening when child fall asleep :)
On 8 Sep 2015 08:19, "Stefan Behnel" <stefan_ml at behnel.de> wrote:

> Fabrizio Messina schrieb am 07.09.2015 um 12:23:
> > On 5 Sep 2015 20:32, "Stefan Behnel" wrote:
> >> It looks like PEP 498 (f-strings) is going to be accepted soon. Anyone
> >> interested in implementing it for Cython?
> >>
> >> https://www.python.org/dev/peps/pep-0498/
> >>
> >> It's certainly a bit of work, but it's also very much Python-level
> >> functionality and most of it can be handled in the parser, so I would
> count
> >> it as an entry-level project for working on Cython. There is already an
> >> initial implementation for CPython (in C) which should provide an
> answer to
> >> the tricky details of the functionality itself, as well as tests for it
> >> (eventually).
> >>
> >> https://bugs.python.org/issue24965
> >>
> >> So, if this feature, and if extending the Cython language and working
> on a
> >> real compiler sounds tempting to you, please speak up on the
> cython-devel
> >> mailing list.
> >
> > Hello, I would like to give a look on that on my spare time.
>
> Cool. If you need any help or have any questions, just reply to this
> thread.
>
> Here's an intro to get you started. It explains how the test suite works
> and how you can figure out how the C code you see is being generated.
>
> https://github.com/cython/cython/wiki/HackerGuide#getting-started
>
> Stefan
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150908/a51c4750/attachment.html>

From jelle.zijlstra at gmail.com  Wed Sep  9 22:58:21 2015
From: jelle.zijlstra at gmail.com (Jelle Zijlstra)
Date: Wed, 9 Sep 2015 13:58:21 -0700
Subject: [Cython] [cython-users] call for contribution: PEP 498 -
	Literal String Interpolation
In-Reply-To: <CAFp3-p8-4MQz7G1dVmFXc3vnmrHGU=rTG+bAMrzR575ZECF+SQ@mail.gmail.com>
References: <55EB352D.4000101@behnel.de>
 <CAFp3-p8-4MQz7G1dVmFXc3vnmrHGU=rTG+bAMrzR575ZECF+SQ@mail.gmail.com>
Message-ID: <CAFp3-p_2HRuTzqLGinrH4qnP2Qr002_-fS+sURbYcNyipyM9Vw@mail.gmail.com>

I would be interested in doing this, although I haven't previously worked
on Cython itself. I'll start looking into it over the weekend.

2015-09-09 7:35 GMT-07:00 Jelle Zijlstra <jelle.zijlstra at gmail.com>:

> I would be interested in doing this, although I haven't previously worked
> on Cython itself. I'll start looking into it over the weekend.
>
> 2015-09-05 11:32 GMT-07:00 Stefan Behnel <stefan_ml at behnel.de>:
>
>> Hi!
>>
>> It looks like PEP 498 (f-strings) is going to be accepted soon. Anyone
>> interested in implementing it for Cython?
>>
>> https://www.python.org/dev/peps/pep-0498/
>>
>> It's certainly a bit of work, but it's also very much Python-level
>> functionality and most of it can be handled in the parser, so I would
>> count
>> it as an entry-level project for working on Cython. There is already an
>> initial implementation for CPython (in C) which should provide an answer
>> to
>> the tricky details of the functionality itself, as well as tests for it
>> (eventually).
>>
>> https://bugs.python.org/issue24965
>>
>> So, if this feature, and if extending the Cython language and working on a
>> real compiler sounds tempting to you, please speak up on the cython-devel
>> mailing list.
>>
>> Stefan
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "cython-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cython-users+unsubscribe at googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150909/564a64fe/attachment.html>

From stefan_ml at behnel.de  Fri Sep 11 10:25:28 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 11 Sep 2015 10:25:28 +0200
Subject: [Cython] Cython 0.23.2 released
Message-ID: <55F28FF8.9090404@behnel.de>

Hi everyone,

here is a second bug-fix release for the 0.23 release series that repairs
several problems and regressions.

https://pypi.python.org/pypi/Cython/0.23.2

Complete changelog follows below. Upgrading is generally recommended.


You can get the signed release from here:

http://cython.org/

http://cython.org/release/Cython-0.23.2.tar.gz

http://cython.org/release/Cython-0.23.2.zip


SHA1 sums:
196bcc55bc82da7e096b28f4eeb848d46a516b21  Cython-0.23.2.tar.gz
7e6aeb8ea81780c65178fa43c9e2b217b2561f8f  Cython-0.23.2.zip

Have fun,

Stefan


0.23.2 (2015-09-11)
===================

Bugs fixed
----------

* Compiler crash when analysing some optimised expressions.

* Coverage plugin was adapted to coverage.py 4.0 beta 2.

* C++ destructor calls could fail when '&' operator is overwritten.

* Incorrect C literal generation for large integers in compile-time
  evaluated DEF expressions and constant folded expressions.

* Byte string constants could end up as Unicode strings when originating
  from compile-time evaluated DEF expressions.

* Invalid C code when caching known builtin methods.
  This fixes ticket 860.

* ``ino_t`` in ``posix.types`` was not declared as ``unsigned``.

* Declarations in ``libcpp/memory.pxd`` were missing ``operator!()``.
  Patch by Leo Razoumov.

* Static cdef methods can now be declared in .pxd files.

From jwilk at jwilk.net  Sat Sep 12 14:59:10 2015
From: jwilk at jwilk.net (Jakub Wilk)
Date: Sat, 12 Sep 2015 14:59:10 +0200
Subject: [Cython] DEF converts byte strings to unicode
In-Reply-To: <55E67E89.3020106@behnel.de>
References: <20150830174400.GA8322@jwilk.net>
 <55E67E89.3020106@behnel.de>
Message-ID: <20150912125910.GA5216@jwilk.net>

I think something is still not quite right in Cython 0.23.2.

Consider this code:

DEF FOO = 'foo'
print type('foo')
print type(FOO)

In Python 3, I get:

<class 'str'>
<class 'bytes'>

-- 
Jakub Wilk

From stefan_ml at behnel.de  Sat Sep 12 18:14:41 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 12 Sep 2015 18:14:41 +0200
Subject: [Cython] DEF converts byte strings to unicode
In-Reply-To: <20150912125910.GA5216@jwilk.net>
References: <20150830174400.GA8322@jwilk.net> <55E67E89.3020106@behnel.de>
 <20150912125910.GA5216@jwilk.net>
Message-ID: <55F44F71.6020203@behnel.de>

Jakub Wilk schrieb am 12.09.2015 um 14:59:
> I think something is still not quite right in Cython 0.23.2.
> 
> Consider this code:
> 
> DEF FOO = 'foo'
> print type('foo')
> print type(FOO)
> 
> In Python 3, I get:
> 
> <class 'str'>
> <class 'bytes'>

Remember that DEF uses compile time evaluation in *Python*. Python does not
have the three string types that Cython has, it has only two: either
str/unicode (Py2) or bytes/str (Py3). If you pass an unprefixed string
through compile time evaluation, it looses the information that it was
unprefixed and turns into a specific Python string object type (i.e. bytes
or unicode), which in this case is bytes, lacking any kind of encoding
information.

Cython follows Py2 semantics by default, so having it turn into a bytes
(i.e. Py2 str) object is actually not wrong. Certainly not more wrong than
a unicode string would be. If you compile in Py3 mode, you should get a
Unicode string.

My general recommendation is to a) avoid DEF, b) avoid DEF for string
values, and c) avoid DEF for unprefixed string values, in that order. But
b) and c) are only for advanced users.

Stefan


From stefan_ml at behnel.de  Sat Sep 12 19:23:26 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 12 Sep 2015 19:23:26 +0200
Subject: [Cython] DEF converts byte strings to unicode
In-Reply-To: <55F44F71.6020203@behnel.de>
References: <20150830174400.GA8322@jwilk.net> <55E67E89.3020106@behnel.de>
 <20150912125910.GA5216@jwilk.net> <55F44F71.6020203@behnel.de>
Message-ID: <55F45F8E.107@behnel.de>

Stefan Behnel schrieb am 12.09.2015 um 18:14:
> Jakub Wilk schrieb am 12.09.2015 um 14:59:
>> I think something is still not quite right in Cython 0.23.2.
>>
>> Consider this code:
>>
>> DEF FOO = 'foo'
>> print type('foo')
>> print type(FOO)
>>
>> In Python 3, I get:
>>
>> <class 'str'>
>> <class 'bytes'>
> 
> Remember that DEF uses compile time evaluation in *Python*. Python does not
> have the three string types that Cython has, it has only two: either
> str/unicode (Py2) or bytes/str (Py3). If you pass an unprefixed string
> through compile time evaluation, it looses the information that it was
> unprefixed and turns into a specific Python string object type (i.e. bytes
> or unicode), which in this case is bytes, lacking any kind of encoding
> information.
> 
> Cython follows Py2 semantics by default, so having it turn into a bytes
> (i.e. Py2 str) object is actually not wrong. Certainly not more wrong than
> a unicode string would be. If you compile in Py3 mode, you should get a
> Unicode string.
> 
> My general recommendation is to a) avoid DEF, b) avoid DEF for string
> values, and c) avoid DEF for unprefixed string values, in that order. But
> b) and c) are only for advanced users.

That being said, always returning a bytes object is actually unhelpful in
Python 3. Let's see if anyone complains if we change that.

https://github.com/cython/cython/commit/ba350910b67db90c14e0aab79eafe7ac1be1d837

Stefan


From jwilk at jwilk.net  Sat Sep 12 19:01:02 2015
From: jwilk at jwilk.net (Jakub Wilk)
Date: Sat, 12 Sep 2015 19:01:02 +0200
Subject: [Cython] DEF converts byte strings to unicode
In-Reply-To: <55F44F71.6020203@behnel.de>
References: <20150830174400.GA8322@jwilk.net> <55E67E89.3020106@behnel.de>
 <20150912125910.GA5216@jwilk.net> <55F44F71.6020203@behnel.de>
Message-ID: <20150912170102.GA8676@jwilk.net>

* Stefan Behnel <stefan_ml at behnel.de>, 2015-09-12, 18:14:
>My general recommendation is to a) avoid DEF, b) avoid DEF for string 
>values, and c) avoid DEF for unprefixed string values, in that order. 
>But b) and c) are only for advanced users.

Fair enough.

NB, there are examples that use DEFs with string values in the 
documentation[0]. You might want to fix them, or add appropriate 
warnings.


[0] $ grep -r "DEF.*=.*['\"]" docs/
docs/src/reference/language_basics.rst:    DEF FavouriteFood = "spam"
docs/src/userguide/language_basics.rst:    DEF FavouriteFood = "spam"

-- 
Jakub Wilk

From jelle.zijlstra at gmail.com  Sun Sep 13 21:00:35 2015
From: jelle.zijlstra at gmail.com (Jelle Zijlstra)
Date: Sun, 13 Sep 2015 12:00:35 -0700
Subject: [Cython] CPython incompatibility in string literal concatenation
Message-ID: <CAFp3-p9pPHcW4N0pq_sa-EJqvLYh+x9ipd2JVOboqHvwvnemHw@mail.gmail.com>

While implementing PEP 498 I found a minor incompatibility between
CPython and Cython: CPython lets you concatenate u-prefixed and
non-prefixed string literals, while Cython throws an error.

jelle at devjelle:~/cython$ cat concat.py
print(u'foo' 'bar')

jelle at devjelle:~/cython$ python concat.py
foobar
jelle at devjelle:~/cython$ cython concat.py

Error compiling Cython file:
------------------------------------------------------------
...
print(u'foo' 'bar')
            ^
------------------------------------------------------------

concat.py:1:13: Cannot mix string literals of different types,
expected u'', got ''


I tried both CPython 2.7 and 3.5. The documentation at
https://docs.python.org/2/reference/lexical_analysis.html#string-literal-concatenation
is ambiguous as to whether this behavior is intentional.

As part of implementing PEP 498, I can make Cython's behavior match
CPython's, unless there's a good reason to keep the incompatibility.

From njs at vorpus.org  Mon Sep 14 14:33:06 2015
From: njs at vorpus.org (Nathaniel Smith)
Date: Mon, 14 Sep 2015 05:33:06 -0700
Subject: [Cython] Is it easy to do an "AST grep" of Cython code?
Message-ID: <CAPJVwBm73_5FcpXoA0w+2j0EeNF_Zpr2LCUBN_X13fUzXcPh0A@mail.gmail.com>

I have a few hundred files worth of Cython code, and I'd like to find
all instances where a variable/expression has a certain cdef class
type. (More specifically, I'd like to find all accesses to the cdef
fields of a particular cdef class, but even just finding all 'cdef
MYTYPE x" and "<MYTYPE>x" statements would probably get me pretty
close.) Is there any easy way to do this?

(The files are "every cython file on github or searchcode.com that
mentions the word "ufunc"", and I'm trying to find code that does
direct field access to the internals of the PyUFuncObject struct.)

-- 
Nathaniel J. Smith -- http://vorpus.org

From jelle.zijlstra at gmail.com  Mon Sep 14 18:36:55 2015
From: jelle.zijlstra at gmail.com (Jelle Zijlstra)
Date: Mon, 14 Sep 2015 09:36:55 -0700
Subject: [Cython] Is it easy to do an "AST grep" of Cython code?
In-Reply-To: <CAPJVwBm73_5FcpXoA0w+2j0EeNF_Zpr2LCUBN_X13fUzXcPh0A@mail.gmail.com>
References: <CAPJVwBm73_5FcpXoA0w+2j0EeNF_Zpr2LCUBN_X13fUzXcPh0A@mail.gmail.com>
Message-ID: <CAFp3-p84ThDbALROhBt3F0eTcyYMJZss4sF80xC4jQZ0kPK7mQ@mail.gmail.com>

This seems doable in principle by using the
Cython.Compiler.Visitor.TreeVisitor class. You'll have to figure out
what Cython AST nodes the things you're looking for correspond to. As
a simple example, this will find all non-def functions in a Cython
file:

from Cython.Compiler import TreeFragment, Visitor

cython_code = unicode(open("filename.pyx").read())

class CdefFunFinder(Visitor.TreeVisitor):
    def visit_Node(self, node):
        self.visitchildren(node)

    def visit_CFuncDefNode(self, node):
        self.visitchildren(node)
        print self.dump_node(node)

CdefFunFinder().visit(TreeFragment.parse_from_strings('<test>',
unicode(cython_code)))

2015-09-14 5:33 GMT-07:00 Nathaniel Smith <njs at vorpus.org>:
> I have a few hundred files worth of Cython code, and I'd like to find
> all instances where a variable/expression has a certain cdef class
> type. (More specifically, I'd like to find all accesses to the cdef
> fields of a particular cdef class, but even just finding all 'cdef
> MYTYPE x" and "<MYTYPE>x" statements would probably get me pretty
> close.) Is there any easy way to do this?
>
> (The files are "every cython file on github or searchcode.com that
> mentions the word "ufunc"", and I'm trying to find code that does
> direct field access to the internals of the PyUFuncObject struct.)
>
> --
> Nathaniel J. Smith -- http://vorpus.org
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel

From stefan_ml at behnel.de  Sat Sep 19 22:07:05 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 19 Sep 2015 22:07:05 +0200
Subject: [Cython] CPython incompatibility in string literal concatenation
In-Reply-To: <CAFp3-p9pPHcW4N0pq_sa-EJqvLYh+x9ipd2JVOboqHvwvnemHw@mail.gmail.com>
References: <CAFp3-p9pPHcW4N0pq_sa-EJqvLYh+x9ipd2JVOboqHvwvnemHw@mail.gmail.com>
Message-ID: <55FDC069.2060803@behnel.de>

Hi,

sorry for the late reply.

Jelle Zijlstra schrieb am 13.09.2015 um 21:00:
> While implementing PEP 498 I found a minor incompatibility between
> CPython and Cython: CPython lets you concatenate u-prefixed and
> non-prefixed string literals, while Cython throws an error.
> 
> jelle at devjelle:~/cython$ cat concat.py
> print(u'foo' 'bar')
> 
> jelle at devjelle:~/cython$ python concat.py
> foobar
> jelle at devjelle:~/cython$ cython concat.py
> 
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> print(u'foo' 'bar')
>             ^
> ------------------------------------------------------------
> 
> concat.py:1:13: Cannot mix string literals of different types,
> expected u'', got ''
> 
> 
> I tried both CPython 2.7 and 3.5. The documentation at
> https://docs.python.org/2/reference/lexical_analysis.html#string-literal-concatenation
> is ambiguous as to whether this behavior is intentional.
> 
> As part of implementing PEP 498, I can make Cython's behavior match
> CPython's, unless there's a good reason to keep the incompatibility.

This obviously works in Py3.3+, where the "u" prefix is simply ignored. It
also works in Py2 because of the implicit promotion from byte strings to
unicode strings.

I'm ok with supporting this in Cython. The string literal parser already
generates a Unicode representation of unprefixed strings which can be used
for concatenation. In fact, the above is currently an explicit error case.

Stefan


From lists at onerussian.com  Mon Sep 21 16:26:56 2015
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Mon, 21 Sep 2015 10:26:56 -0400
Subject: [Cython] ct_DEF.__test__.large_nums doctest failing in 32bit
Message-ID: <20150921142656.GL17366@onerussian.com>

was reported in Debian against 0.23.2 so tried blindly freshier snapshot
from 0.23.x branch and that one is no go too:

======================================================================
FAIL: large_nums (line 95) (ct_DEF.__test__)
Doctest: ct_DEF.__test__.large_nums (line 95)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for ct_DEF.__test__.large_nums (line 95)
  File "/tmp/buildd/cython-0.23.2+git12-g2c9d175/build/work-dir/run/cpp/ct_DEF/ct_DEF.so", line unknown line number, in large_nums (line 95)

----------------------------------------------------------------------
File "/tmp/buildd/cython-0.23.2+git12-g2c9d175/build/work-dir/run/cpp/ct_DEF/ct_DEF.so", line ?, in ct_DEF.__test__.large_nums (line 95)
Failed example:
    print_large_number(n64)
Expected:
    -4294967295
Got:
    1



-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

From stefan_ml at behnel.de  Tue Sep 22 16:03:13 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 22 Sep 2015 16:03:13 +0200
Subject: [Cython] ct_DEF.__test__.large_nums doctest failing in 32bit
In-Reply-To: <20150921142656.GL17366@onerussian.com>
References: <20150921142656.GL17366@onerussian.com>
Message-ID: <56015FA1.8080002@behnel.de>

Yaroslav Halchenko schrieb am 21.09.2015 um 16:26:
> was reported in Debian against 0.23.2 so tried blindly freshier snapshot
> from 0.23.x branch and that one is no go too:
> 
> ======================================================================
> FAIL: large_nums (line 95) (ct_DEF.__test__)
> Doctest: ct_DEF.__test__.large_nums (line 95)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/doctest.py", line 2226, in runTest
>     raise self.failureException(self.format_failure(new.getvalue()))
> AssertionError: Failed doctest test for ct_DEF.__test__.large_nums (line 95)
>   File "/tmp/buildd/cython-0.23.2+git12-g2c9d175/build/work-dir/run/cpp/ct_DEF/ct_DEF.so", line unknown line number, in large_nums (line 95)
> 
> ----------------------------------------------------------------------
> File "/tmp/buildd/cython-0.23.2+git12-g2c9d175/build/work-dir/run/cpp/ct_DEF/ct_DEF.so", line ?, in ct_DEF.__test__.large_nums (line 95)
> Failed example:
>     print_large_number(n64)
> Expected:
>     -4294967295
> Got:
>     1

I cannot reproduce this locally (even tried a 32bit docker container), so I
blindly changed a couple of minor things in that branch. Could you give it
another try?

If that didn't help, here is a patch that adds a longness suffix to plain
integer literals based on their type (e.g. found after coercions). I'm not
sure yet if that's really a good idea, but would be helpful to know if it
fixes this test. Could you try both and report back?

Thanks!

Stefan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: guess_longness.patch
Type: text/x-patch
Size: 1036 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150922/e1103da3/attachment.bin>

From lists at onerussian.com  Wed Sep 23 16:08:45 2015
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Wed, 23 Sep 2015 10:08:45 -0400
Subject: [Cython] ct_DEF.__test__.large_nums doctest failing in 32bit
In-Reply-To: <56015FA1.8080002@behnel.de>
References: <20150921142656.GL17366@onerussian.com>
 <56015FA1.8080002@behnel.de>
Message-ID: <20150923140845.GM17366@onerussian.com>


On Tue, 22 Sep 2015, Stefan Behnel wrote:
> > Expected:
> >     -4294967295
> > Got:
> >     1

> I cannot reproduce this locally (even tried a 32bit docker container), so I
> blindly changed a couple of minor things in that branch. Could you give it
> another try?

> If that didn't help, here is a patch that adds a longness suffix to plain
> integer literals based on their type (e.g. found after coercions). I'm not
> sure yet if that's really a good idea, but would be helpful to know if it
> fixes this test. Could you try both and report back?

thanks -- will do!
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

From lists at onerussian.com  Thu Sep 24 04:43:54 2015
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Wed, 23 Sep 2015 22:43:54 -0400
Subject: [Cython] ct_DEF.__test__.large_nums doctest failing in 32bit
In-Reply-To: <20150923140845.GM17366@onerussian.com>
References: <20150921142656.GL17366@onerussian.com>
 <56015FA1.8080002@behnel.de>
 <20150923140845.GM17366@onerussian.com>
Message-ID: <20150924024354.GN17366@onerussian.com>


On Wed, 23 Sep 2015, Yaroslav Halchenko wrote:


> On Tue, 22 Sep 2015, Stefan Behnel wrote:
> > > Expected:
> > >     -4294967295
> > > Got:
> > >     1

> > I cannot reproduce this locally (even tried a 32bit docker container), so I
> > blindly changed a couple of minor things in that branch. Could you give it
> > another try?

> > If that didn't help, here is a patch that adds a longness suffix to plain
> > integer literals based on their type (e.g. found after coercions). I'm not
> > sure yet if that's really a good idea, but would be helpful to know if it
> > fixes this test. Could you try both and report back?

> thanks -- will do!

reporting: it did need guess_longness.patch to pass

Thanks!
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

From mirth.hickford at gmail.com  Sun Sep 27 21:30:21 2015
From: mirth.hickford at gmail.com (Mirth Hickford)
Date: Sun, 27 Sep 2015 20:30:21 +0100
Subject: [Cython] Publish Windows wheels to PyPI
Message-ID: <CAGJzqsnpJYhCunVP=k26obEpmcbRQXxK8kXhT_LFzOEKsdNEcA@mail.gmail.com>

Hi Cython. Please consider publishing wheels for Windows to PyPI [1].
Wheels [2] are a package format that installs more reliably than source
distributions, especially on Windows.

Meanwhile a workaround is to download wheels from Christoph Gohlke's site
[3] a godsend.

-M

[1] https://pypi.python.org/pypi/Cython/
[2] http://pythonwheels.com/
[3] http://www.lfd.uci.edu/~gohlke/pythonlibs/#cython
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150927/71192239/attachment.html>

From thecere at gmail.com  Mon Sep 28 09:22:51 2015
From: thecere at gmail.com (Dirk Rothe)
Date: Mon, 28 Sep 2015 09:22:51 +0200
Subject: [Cython] Publish Windows wheels to PyPI
In-Reply-To: <CAGJzqsnpJYhCunVP=k26obEpmcbRQXxK8kXhT_LFzOEKsdNEcA@mail.gmail.com>
References: <CAGJzqsnpJYhCunVP=k26obEpmcbRQXxK8kXhT_LFzOEKsdNEcA@mail.gmail.com>
Message-ID: <op.x5nv4dbd611alc@cere-base.semantics.de>

On Sun, 27 Sep 2015 21:30:21 +0200, Mirth Hickford  
<mirth.hickford at gmail.com> wrote:

> Hi Cython. Please consider publishing wheels for Windows to PyPI [1].
> Wheels [2] are a package format that installs more reliably than source
> distributions, especially on Windows.
>
> Meanwhile a workaround is to download wheels from Christoph Gohlke's site
> [3] a godsend.

Yeah, he's awesome. Excellent work for at least 10 years now. Thanks!

Cython is awesome too, btw ;)

> -M
>
> [1] https://pypi.python.org/pypi/Cython/
> [2] http://pythonwheels.com/
> [3] http://www.lfd.uci.edu/~gohlke/pythonlibs/#cython

From robertwb at gmail.com  Mon Sep 28 23:15:00 2015
From: robertwb at gmail.com (Robert Bradshaw)
Date: Mon, 28 Sep 2015 14:15:00 -0700
Subject: [Cython] Publish Windows wheels to PyPI
In-Reply-To: <op.x5nv4dbd611alc@cere-base.semantics.de>
References: <CAGJzqsnpJYhCunVP=k26obEpmcbRQXxK8kXhT_LFzOEKsdNEcA@mail.gmail.com>
 <op.x5nv4dbd611alc@cere-base.semantics.de>
Message-ID: <CADiQ+QDV4+sYWj8ik+SyAmbeVcbkKQN01ZOVHD0R0SDboaL6Uw@mail.gmail.com>

On Mon, Sep 28, 2015 at 12:22 AM, Dirk Rothe <thecere at gmail.com> wrote:
> On Sun, 27 Sep 2015 21:30:21 +0200, Mirth Hickford
> <mirth.hickford at gmail.com> wrote:
>
>> Hi Cython. Please consider publishing wheels for Windows to PyPI [1].
>> Wheels [2] are a package format that installs more reliably than source
>> distributions, especially on Windows.

I don't own a Windows machine (and nor do many other Cython devs) so
I'm not sure how to do this (let alone navigate the issue of choosing
the right compiler, etc.) But this'd be a great area for a Window
Cython user to contribute.

https://github.com/cython/cython/pull/389 was recently merged.

>> Meanwhile a workaround is to download wheels from Christoph Gohlke's site
>> [3] a godsend.
>
> Yeah, he's awesome. Excellent work for at least 10 years now. Thanks!

+1, thanks Christoph!

> Cython is awesome too, btw ;)
>
>> -M
>>
>> [1] https://pypi.python.org/pypi/Cython/
>> [2] http://pythonwheels.com/
>> [3] http://www.lfd.uci.edu/~gohlke/pythonlibs/#cython
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel

From ian.h.bell at gmail.com  Mon Sep 28 23:28:04 2015
From: ian.h.bell at gmail.com (Ian Bell)
Date: Mon, 28 Sep 2015 15:28:04 -0600
Subject: [Cython] Publish Windows wheels to PyPI
In-Reply-To: <CADiQ+QDV4+sYWj8ik+SyAmbeVcbkKQN01ZOVHD0R0SDboaL6Uw@mail.gmail.com>
References: <CAGJzqsnpJYhCunVP=k26obEpmcbRQXxK8kXhT_LFzOEKsdNEcA@mail.gmail.com>
 <op.x5nv4dbd611alc@cere-base.semantics.de>
 <CADiQ+QDV4+sYWj8ik+SyAmbeVcbkKQN01ZOVHD0R0SDboaL6Uw@mail.gmail.com>
Message-ID: <CAJQnXJf_o9xtbajehwFtfVCXzG=BgLc4K3f2jygY4gvhj1Z1tA@mail.gmail.com>

I volunteered to do this quite some time ago, but at the time there wasn't
much interest.  The offer still stands though.  I have a number of build
machines that I could use and are properly configured with all the python
versions and compilers needed.  I've released other python packages in the
same way before.

Ian

On Mon, Sep 28, 2015 at 3:15 PM, Robert Bradshaw <robertwb at gmail.com> wrote:

> On Mon, Sep 28, 2015 at 12:22 AM, Dirk Rothe <thecere at gmail.com> wrote:
> > On Sun, 27 Sep 2015 21:30:21 +0200, Mirth Hickford
> > <mirth.hickford at gmail.com> wrote:
> >
> >> Hi Cython. Please consider publishing wheels for Windows to PyPI [1].
> >> Wheels [2] are a package format that installs more reliably than source
> >> distributions, especially on Windows.
>
> I don't own a Windows machine (and nor do many other Cython devs) so
> I'm not sure how to do this (let alone navigate the issue of choosing
> the right compiler, etc.) But this'd be a great area for a Window
> Cython user to contribute.
>
> https://github.com/cython/cython/pull/389 was recently merged.
>
> >> Meanwhile a workaround is to download wheels from Christoph Gohlke's
> site
> >> [3] a godsend.
> >
> > Yeah, he's awesome. Excellent work for at least 10 years now. Thanks!
>
> +1, thanks Christoph!
>
> > Cython is awesome too, btw ;)
> >
> >> -M
> >>
> >> [1] https://pypi.python.org/pypi/Cython/
> >> [2] http://pythonwheels.com/
> >> [3] http://www.lfd.uci.edu/~gohlke/pythonlibs/#cython
> >
> > _______________________________________________
> > cython-devel mailing list
> > cython-devel at python.org
> > https://mail.python.org/mailman/listinfo/cython-devel
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150928/b800f50d/attachment.html>

From mirth.hickford at gmail.com  Tue Sep 29 00:31:29 2015
From: mirth.hickford at gmail.com (Mirth Hickford)
Date: Mon, 28 Sep 2015 23:31:29 +0100
Subject: [Cython] Publish Windows wheels to PyPI
In-Reply-To: <CADiQ+QDV4+sYWj8ik+SyAmbeVcbkKQN01ZOVHD0R0SDboaL6Uw@mail.gmail.com>
References: <CAGJzqsnpJYhCunVP=k26obEpmcbRQXxK8kXhT_LFzOEKsdNEcA@mail.gmail.com>
 <op.x5nv4dbd611alc@cere-base.semantics.de>
 <CADiQ+QDV4+sYWj8ik+SyAmbeVcbkKQN01ZOVHD0R0SDboaL6Uw@mail.gmail.com>
Message-ID: <CAGJzqsnFbqkYTLBYU_QZ+jzKjWAQUMOrEZnWC+zp6OLpiDE8GQ@mail.gmail.com>

> https://github.com/cython/cython/pull/389 was recently merged.

Hum there aren't yet any builds at
https://ci.appveyor.com/project/cython/cython

I think someone (with rights to the github repo) still needs to create an
account at appveyor.com and flip a switch. See
http://help.appveyor.com/discussions/questions/1154-appveyor-account-for-github-organizations
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150928/143031cb/attachment.html>

From robertwb at gmail.com  Tue Sep 29 01:32:57 2015
From: robertwb at gmail.com (Robert Bradshaw)
Date: Mon, 28 Sep 2015 16:32:57 -0700
Subject: [Cython] Publish Windows wheels to PyPI
In-Reply-To: <CAGJzqsnFbqkYTLBYU_QZ+jzKjWAQUMOrEZnWC+zp6OLpiDE8GQ@mail.gmail.com>
References: <CAGJzqsnpJYhCunVP=k26obEpmcbRQXxK8kXhT_LFzOEKsdNEcA@mail.gmail.com>
 <op.x5nv4dbd611alc@cere-base.semantics.de>
 <CADiQ+QDV4+sYWj8ik+SyAmbeVcbkKQN01ZOVHD0R0SDboaL6Uw@mail.gmail.com>
 <CAGJzqsnFbqkYTLBYU_QZ+jzKjWAQUMOrEZnWC+zp6OLpiDE8GQ@mail.gmail.com>
Message-ID: <CADiQ+QC+RB9m4KwGmpyg+6my8jwAJFj-JsGXkWHExub0kxoYdQ@mail.gmail.com>

OK, I created a cython account, but
https://ci.appveyor.com/project/cython/cython . I'm happy to add
anyone as an appveyer "contributor" who could help get this going.

On Mon, Sep 28, 2015 at 3:31 PM, Mirth Hickford
<mirth.hickford at gmail.com> wrote:
>> https://github.com/cython/cython/pull/389 was recently merged.
>
> Hum there aren't yet any builds at
> https://ci.appveyor.com/project/cython/cython
>
> I think someone (with rights to the github repo) still needs to create an
> account at appveyor.com and flip a switch. See
> http://help.appveyor.com/discussions/questions/1154-appveyor-account-for-github-organizations
>
>
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>

From stefan_ml at behnel.de  Tue Sep 29 11:57:21 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 29 Sep 2015 11:57:21 +0200
Subject: [Cython] Cython 0.23.3 released
Message-ID: <560A6081.1050708@behnel.de>

Hi everyone,

here is a third bug-fix release for the 0.23 release series that repairs
some further problems and regressions.

https://pypi.python.org/pypi/Cython/0.23.3

Complete changelog follows below. Upgrading is generally recommended.


You can get the signed release from here:

http://cython.org/

http://cython.org/release/Cython-0.23.3.tar.gz

http://cython.org/release/Cython-0.23.3.zip


SHA1 sums:
d5592dc3d529c55a5ef95346caccf11c556993bd  Cython-0.23.3.tar.gz
25c15d661fa2632be1f336f591ef989a76512fe3  Cython-0.23.3.zip

Have fun,

Stefan


0.23.3 (2015-09-29)
===================

Bugs fixed
----------

* Invalid C code for some builtin methods.  This fixes ticket 856 again.

* Incorrect C code in helper functions for PyLong conversion and string
  decoding.  This fixes ticket 863, ticket 864 and ticket 865.
  Original patch by Nikolaus Rath.

* Large folded or inserted integer constants could use too small C
  integer types and thus trigger a value wrap-around.

Other changes
-------------

* The coroutine and generator types of Cython now also register directly
  with the ``Coroutine`` and ``Generator`` ABCs in the ``backports_abc``
  module if it can be imported.  This fixes ticket 870.

From christopher.hogan at intel.com  Wed Sep 30 00:03:00 2015
From: christopher.hogan at intel.com (Hogan, Christopher)
Date: Tue, 29 Sep 2015 22:03:00 +0000
Subject: [Cython] Bug: llabs not defined in vs2008
Message-ID: <18A68EC651142A408A0A6AFDE7D1FC895C149C@ORSMSX103.amr.corp.intel.com>

Hello,

I was trying to compile numpy v1.9.2 with Visual Studio 2008 and Cython 0.23.1.  This worked fine with Cython 0.22.x, but in 0.23.1 there are some changes in Cython/Utility/TypeConversion.c that seem to cause this error:

mtrand.obj : error LNK2019: unresolved external symbol llabs referenced in function __pyx_pf_6mtrand_11RandomState_24 choice

llabs is not available in vs2008, but it ends up taking that "#elif" branch in TypeConversion.c and defining __Pyx_sst_abs to be llabs.  I was able to fix this by swapping the order of the "#elif defined (_MSC_VER) ..."  block with the "#elif defined (__STDC_VERSION__) ... " block.  That probably won't fix the problem in all cases though.

Thanks,

Chris Hogan
Scripting Tools Engineer
Scripting, Analyzers and Tools
Intel Corporation

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20150929/b278174e/attachment.html>

From stefan_ml at behnel.de  Wed Sep 30 09:13:03 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 30 Sep 2015 09:13:03 +0200
Subject: [Cython] Bug: llabs not defined in vs2008
In-Reply-To: <18A68EC651142A408A0A6AFDE7D1FC895C149C@ORSMSX103.amr.corp.intel.com>
References: <18A68EC651142A408A0A6AFDE7D1FC895C149C@ORSMSX103.amr.corp.intel.com>
Message-ID: <560B8B7F.6030401@behnel.de>

Hi!

None of the core developers uses MSVC, so we generally rely on Windows
users to do their own testing and provide fixes.

Hogan, Christopher schrieb am 30.09.2015 um 00:03:
> I was trying to compile numpy v1.9.2 with Visual Studio 2008 and Cython
> 0.23.1.  This worked fine with Cython 0.22.x, but in 0.23.1 there are
> some changes in Cython/Utility/TypeConversion.c that seem to cause this
> error:
> 
> mtrand.obj : error LNK2019: unresolved external symbol llabs referenced
> in function __pyx_pf_6mtrand_11RandomState_24 choice
> 
> llabs is not available in vs2008, but it ends up taking that "#elif"
> branch in TypeConversion.c and defining __Pyx_sst_abs to be llabs.  I
> was able to fix this by swapping the order of the "#elif defined
> (_MSC_VER) ..."  block with the "#elif defined (__STDC_VERSION__) ... "
> block.

So, what you are saying is that VS-2008 is not C99 standards compliant
(what a surprise) but claims to be? Are you passing any C compiler flags
that requests C99 compliance?

The last changes in this area were here:

https://github.com/cython/cython/pull/403

The contributor claimed to be using MSVC 2008, too, so I wonder why you run
into this problem but he didn't.


> That probably won't fix the problem in all cases though.

To me, your proposed change seems a reasonable thing to do. What cases are
you thinking of?

Stefan

From cgohlke at uci.edu  Wed Sep 30 09:53:29 2015
From: cgohlke at uci.edu (Christoph Gohlke)
Date: Wed, 30 Sep 2015 00:53:29 -0700
Subject: [Cython] Bug: llabs not defined in vs2008
In-Reply-To: <560B8B7F.6030401@behnel.de>
References: <18A68EC651142A408A0A6AFDE7D1FC895C149C@ORSMSX103.amr.corp.intel.com>
 <560B8B7F.6030401@behnel.de>
Message-ID: <560B94F9.70907@uci.edu>

Hi,

On 9/30/2015 12:13 AM, Stefan Behnel wrote:
> Hi!
>
> None of the core developers uses MSVC, so we generally rely on Windows
> users to do their own testing and provide fixes.
>
> Hogan, Christopher schrieb am 30.09.2015 um 00:03:
>> I was trying to compile numpy v1.9.2 with Visual Studio 2008 and Cython
>> 0.23.1.  This worked fine with Cython 0.22.x, but in 0.23.1 there are
>> some changes in Cython/Utility/TypeConversion.c that seem to cause this
>> error:
>>
>> mtrand.obj : error LNK2019: unresolved external symbol llabs referenced
>> in function __pyx_pf_6mtrand_11RandomState_24 choice
>>
>> llabs is not available in vs2008, but it ends up taking that "#elif"
>> branch in TypeConversion.c and defining __Pyx_sst_abs to be llabs.  I
>> was able to fix this by swapping the order of the "#elif defined
>> (_MSC_VER) ..."  block with the "#elif defined (__STDC_VERSION__) ... "
>> block.
>
> So, what you are saying is that VS-2008 is not C99 standards compliant
> (what a surprise) but claims to be? Are you passing any C compiler flags
> that requests C99 compliance?
>
> The last changes in this area were here:
>
> https://github.com/cython/cython/pull/403
>
> The contributor claimed to be using MSVC 2008, too, so I wonder why you run
> into this problem but he didn't.


I can't reproduce this either. Probably __STDC_VERSION__ is somehow 
defined on Christopher's system.

Christoph


>
>
>> That probably won't fix the problem in all cases though.
>
> To me, your proposed change seems a reasonable thing to do. What cases are
> you thinking of?
>
> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
>

From stefan_ml at behnel.de  Wed Sep 30 09:51:26 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 30 Sep 2015 09:51:26 +0200
Subject: [Cython] Non-type template parameters
In-Reply-To: <CAPWbfpTv9MZ1G2qoA8OqW7xp_ObegONypXyzqNSGBXZdCQLqPg@mail.gmail.com>
References: <CAPWbfpQcXDUq54y-3dYKMtj7W0ZNh=p4gYacg-NJvewP+p=qug@mail.gmail.com>
 <CADiQ+QAZ5RMh9WaX6911-zzVHq=ihTbfh7ykTF+kjvEM3kLgmw@mail.gmail.com>
 <CAPWbfpSRPzOd+2CuyvzxUG8LM7mbFfWa5-WOr1zRSxwhgP13Kw@mail.gmail.com>
 <CADiQ+QAjesGL45VgsQ4Rxb_n83_=8zwrvxjRysfpg7GDQgSGGg@mail.gmail.com>
 <CAPWbfpTv9MZ1G2qoA8OqW7xp_ObegONypXyzqNSGBXZdCQLqPg@mail.gmail.com>
Message-ID: <560B947E.1020207@behnel.de>

Ian Henriksen schrieb am 29.07.2015 um 23:56:
> On Tue, Jul 28, 2015 at 12:37 PM Robert Bradshaw wrote:
>> On Tue, Jul 28, 2015 at 11:18 AM, Ian Henriksen wrote:
>>> On Tue, Jul 28, 2015 at 1:36 AM Robert Bradshaw wrote:
>>>> On Mon, Jul 20, 2015 at 2:06 PM, Ian Henriksen wrote:
>>>>> I've spent a little time working on adding support for non-type
>>>>> template parameters. In doing this, it has been very easy to add
>>>>> support for doing something like the following:
>>>>>
>>>>> cdef extern from "add_const.hpp" nogil:
>>>>>     int myfunc1[i](int)
>>>>> def test():
>>>>>     print myfunc1[2](a)
>>>>>
>>>>> The downside of this is that it does not let the Cython compiler
>>>>> distinguish between different kinds of template parameters.
>>>>>
>>>>> Stricter checking could be added using a syntax like this:
>>>>>
>>>>> cdef extern from "add_const.hpp" nogil:
>>>>>     int myfunc1[int i](int)
>>>>> def test():
>>>>>     print myfunc1[2](a)
>>>>>
>>>>> The downsides here are that the syntax doesn't really match the
>>>>> existing template syntax. It will also complicate the Cython codebase
>>>>> since we'll have to go to greater lengths to allow or disallow all the
>>>>> different special cases for templates.
>>>>>
>>>>> Which version would be preferable?
>>>>
>>>> I think I'd prefer the [int i] syntax. Hopefully it shouldn't
>>>> complicate things too much.
>>>
>>> Okay, I think I see a way to make that work. On the other hand, since
>>> there
>>> weren't any replies here, I've already nearly finished implementing the
>>> first
>>> syntax. I'll spend another hour or two finishing it off later today and
>>> submit a PR
>>> so you can look it over. I originally favored the first syntax because it
>>> minimizes
>>> the number of fancy template features (SFINAE, for example) we have to
>>> worry about on the Cython end. I'm still open to discuss it though.
>>
>> I think this falls into the "explicit is better than implicit" bucket.
>> That and getting obscure template errors that could have been caught
>> at Cython compile time will be very nice.
>>
>>>>> On a similar note, for variadic templates, would we prefer something
>>>>> like
>>>>>
>>>>> cdef extern from "my_variadic.hpp" nogil:
>>>>>     T myfunc2[T,...](T, ...)
>>>>>
>>>>> or something like:
>>>>>
>>>>> cdef extern from "my_variadic.hpp" nogil:
>>>>>     T myfunc2[T, Types...](T, Types... args)
>>>>>
>>>>> Again, the latter syntax is more explicit, but it will require much
>>>>> more
>>>>> complicated code in Cython. It also doesn't match the existing syntax
>>>>> very well. The former syntax matches the existing syntax for templates
>>>>> better, but will make it hard for Cython to raise errors early on in
>>>>> compilation.
>>>>
>>>> Hmm... this is a tougher call. Let's go with the former for now.
>>>
>>>
>>> I like the former a lot more. It will keep the syntax simpler on our end
>>> and I
>>> haven't been able to find any case that it doesn't cover. This will also
>>> be significantly easier to implement. I'll take a look at it soon.
>>
>> Sounds good.
>>
>> There's also the question of default arguments. Let's adopt the [type
>> T = *] syntax for that.
> 
> Yep, I can see why that syntax makes sense. I'm a little worried I may not
> be able
> to finish adding all of this in the near future. Adding all the of this type
> checking for templates goes a long way toward implementing the full syntax
> for declaring them. That said, it seems sensible to catch errors early if
> possible.

I'd like to see this feature merged rather sooner than later. Would it help
if we only require a type to be there (so that people get the syntax right
in their code) but do not validate it for now?

Stefan


From stefan_ml at behnel.de  Wed Sep 30 09:20:33 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 30 Sep 2015 09:20:33 +0200
Subject: [Cython] Bug: llabs not defined in vs2008
In-Reply-To: <560B8B7F.6030401@behnel.de>
References: <18A68EC651142A408A0A6AFDE7D1FC895C149C@ORSMSX103.amr.corp.intel.com>
 <560B8B7F.6030401@behnel.de>
Message-ID: <560B8D41.6000202@behnel.de>

Stefan Behnel schrieb am 30.09.2015 um 09:13:
> Hogan, Christopher schrieb am 30.09.2015 um 00:03:
>> I was trying to compile numpy v1.9.2 with Visual Studio 2008 and Cython
>> 0.23.1.  This worked fine with Cython 0.22.x, but in 0.23.1 there are
>> some changes in Cython/Utility/TypeConversion.c that seem to cause this
>> error:
>>
>> mtrand.obj : error LNK2019: unresolved external symbol llabs referenced
>> in function __pyx_pf_6mtrand_11RandomState_24 choice
>>
>> llabs is not available in vs2008, but it ends up taking that "#elif"
>> branch in TypeConversion.c and defining __Pyx_sst_abs to be llabs.  I
>> was able to fix this by swapping the order of the "#elif defined
>> (_MSC_VER) ..."  block with the "#elif defined (__STDC_VERSION__) ... "
>> block.
> 
> So, what you are saying is that VS-2008 is not C99 standards compliant
> (what a surprise) but claims to be? Are you passing any C compiler flags
> that requests C99 compliance?
> 
> The last changes in this area were here:
> 
> https://github.com/cython/cython/pull/403
> 
> The contributor claimed to be using MSVC 2008, too, so I wonder why you run
> into this problem but he didn't.
> 
> 
>> That probably won't fix the problem in all cases though.
> 
> To me, your proposed change seems a reasonable thing to do. What cases are
> you thinking of?

I applied your change for now, it's in the 0.23.x. maintenance branch:

https://github.com/cython/cython/commit/8bf936e5449592b8b69dd56c09640db8c0ab12ff

Stefan


From christopher.hogan at intel.com  Wed Sep 30 16:21:10 2015
From: christopher.hogan at intel.com (Hogan, Christopher)
Date: Wed, 30 Sep 2015 14:21:10 +0000
Subject: [Cython] Bug: llabs not defined in vs2008
Message-ID: <18A68EC651142A408A0A6AFDE7D1FC895C159B@ORSMSX103.amr.corp.intel.com>


> So, what you are saying is that VS-2008 is not C99 standards compliant (what a surprise) but claims to be?
> Are you passing any C compiler flags that requests C99 compliance?

We have to use the flag /Qstd=c99.  I guess it's more a Microsoft problem than anything.

> To me, your proposed change seems a reasonable thing to do. What cases are you thinking of?

If someone is using any VS version > 2008, then _abs64 will be defined instead of llabs.  I'm not sure if one is preferable to the other though.  
   
> I applied your change for now, it's in the 0.23.x. maintenance branch:

Thank you.

Chris Hogan
Scripting Tools Engineer
Scripting, Analyzers and Tools
Intel Corporation