[Python-checkins] cpython: Implement PEP 380 - 'yield from' (closes #11682)
nick.coghlan
python-checkins at python.org
Fri Jan 13 12:43:53 CET 2012
http://hg.python.org/cpython/rev/d64ac9ab4cd0
changeset: 74356:d64ac9ab4cd0
user: Nick Coghlan <ncoghlan at gmail.com>
date: Fri Jan 13 21:43:40 2012 +1000
summary:
Implement PEP 380 - 'yield from' (closes #11682)
files:
Doc/library/dis.rst | 7 +
Doc/library/exceptions.rst | 11 +-
Doc/reference/expressions.rst | 30 ++-
Doc/reference/simple_stmts.rst | 24 +-
Doc/whatsnew/3.3.rst | 17 +-
Grammar/Grammar | 5 +-
Include/Python-ast.h | 6 +-
Include/frameobject.h | 43 ++--
Include/genobject.h | 21 +-
Include/graminit.h | 1 +
Include/opcode.h | 169 ++++++++--------
Include/pyerrors.h | 7 +
Lib/opcode.py | 1 +
Lib/test/test_ast.py | 3 +-
Lib/test/test_generators.py | 42 ----
Lib/test/test_grammar.py | 32 +++
Lib/test/test_parser.py | 5 +-
Lib/test/test_sys.py | 2 +-
Misc/ACKS | 2 +
Misc/NEWS | 4 +
Modules/parsermodule.c | 82 +++++--
Objects/abstract.c | 2 -
Objects/exceptions.c | 66 ++++++-
Objects/frameobject.c | 15 +-
Objects/genobject.c | 219 +++++++++++++++++++-
Parser/Python.asdl | 2 +-
Python/Python-ast.c | 27 ++-
Python/ast.c | 21 +-
Python/ceval.c | 46 ++++
Python/compile.c | 8 +-
Python/graminit.c | 190 +++++++++--------
Python/opcode_targets.h | 2 +-
Python/symtable.c | 19 -
33 files changed, 791 insertions(+), 340 deletions(-)
diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst
--- a/Doc/library/dis.rst
+++ b/Doc/library/dis.rst
@@ -431,6 +431,13 @@
Pops ``TOS`` and yields it from a :term:`generator`.
+.. opcode:: YIELD_FROM
+
+ Pops ``TOS`` and delegates to it as a subiterator from a :term:`generator`.
+
+ .. versionadded:: 3.3
+
+
.. opcode:: IMPORT_STAR
Loads all symbols not starting with ``'_'`` directly from the module TOS to the
diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst
--- a/Doc/library/exceptions.rst
+++ b/Doc/library/exceptions.rst
@@ -250,7 +250,16 @@
.. exception:: StopIteration
Raised by built-in function :func:`next` and an :term:`iterator`\'s
- :meth:`__next__` method to signal that there are no further values.
+ :meth:`__next__` method to signal that there are no further items to be
+ produced by the iterator.
+
+ The exception object has a single attribute :attr:`value`, which is
+ given as an argument when constructing the exception, and defaults
+ to :const:`None`.
+
+ When a generator function returns, a new :exc:`StopIteration` instance is
+ raised, and the value returned by the function is used as the
+ :attr:`value` parameter to the constructor of the exception.
.. exception:: SyntaxError
diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst
--- a/Doc/reference/expressions.rst
+++ b/Doc/reference/expressions.rst
@@ -318,7 +318,7 @@
.. productionlist::
yield_atom: "(" `yield_expression` ")"
- yield_expression: "yield" [`expression_list`]
+ yield_expression: "yield" [`expression_list` | "from" `expression`]
The :keyword:`yield` expression is only used when defining a generator function,
and can only be used in the body of a function definition. Using a
@@ -336,7 +336,10 @@
the generator's methods, the function can proceed exactly as if the
:keyword:`yield` expression was just another external call. The value of the
:keyword:`yield` expression after resuming depends on the method which resumed
-the execution.
+the execution. If :meth:`__next__` is used (typically via either a
+:keyword:`for` or the :func:`next` builtin) then the result is :const:`None`,
+otherwise, if :meth:`send` is used, then the result will be the value passed
+in to that method.
.. index:: single: coroutine
@@ -346,12 +349,29 @@
where should the execution continue after it yields; the control is always
transferred to the generator's caller.
-The :keyword:`yield` statement is allowed in the :keyword:`try` clause of a
+:keyword:`yield` expressions are allowed in the :keyword:`try` clause of a
:keyword:`try` ... :keyword:`finally` construct. If the generator is not
resumed before it is finalized (by reaching a zero reference count or by being
garbage collected), the generator-iterator's :meth:`close` method will be
called, allowing any pending :keyword:`finally` clauses to execute.
+When ``yield from expression`` is used, it treats the supplied expression as
+a subiterator. All values produced by that subiterator are passed directly
+to the caller of the current generator's methods. Any values passed in with
+:meth:`send` and any exceptions passed in with :meth:`throw` are passed to
+the underlying iterator if it has the appropriate methods. If this is not the
+case, then :meth:`send` will raise :exc:`AttributeError` or :exc:`TypeError`,
+while :meth:`throw` will just raise the passed in exception immediately.
+
+When the underlying iterator is complete, the :attr:`~StopIteration.value`
+attribute of the raised :exc:`StopIteration` instance becomes the value of
+the yield expression. It can be either set explicitly when raising
+:exc:`StopIteration`, or automatically when the sub-iterator is a generator
+(by returning a value from the sub-generator).
+
+The parentheses can be omitted when the :keyword:`yield` expression is the
+sole expression on the right hand side of an assignment statement.
+
.. index:: object: generator
The following generator's methods can be used to control the execution of a
@@ -444,6 +464,10 @@
The proposal to enhance the API and syntax of generators, making them
usable as simple coroutines.
+ :pep:`0380` - Syntax for Delegating to a Subgenerator
+ The proposal to introduce the :token:`yield_from` syntax, making delegation
+ to sub-generators easy.
+
.. _primaries:
diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst
--- a/Doc/reference/simple_stmts.rst
+++ b/Doc/reference/simple_stmts.rst
@@ -425,10 +425,10 @@
:keyword:`finally` clause, that :keyword:`finally` clause is executed before
really leaving the function.
-In a generator function, the :keyword:`return` statement is not allowed to
-include an :token:`expression_list`. In that context, a bare :keyword:`return`
-indicates that the generator is done and will cause :exc:`StopIteration` to be
-raised.
+In a generator function, the :keyword:`return` statement indicates that the
+generator is done and will cause :exc:`StopIteration` to be raised. The returned
+value (if any) is used as an argument to construct :exc:`StopIteration` and
+becomes the :attr:`StopIteration.value` attribute.
.. _yield:
@@ -450,6 +450,7 @@
and is only used in the body of the generator function. Using a :keyword:`yield`
statement in a function definition is sufficient to cause that definition to
create a generator function instead of a normal function.
+
When a generator function is called, it returns an iterator known as a generator
iterator, or more commonly, a generator. The body of the generator function is
executed by calling the :func:`next` function on the generator repeatedly until
@@ -469,14 +470,25 @@
garbage collected), the generator-iterator's :meth:`close` method will be
called, allowing any pending :keyword:`finally` clauses to execute.
+When ``yield from expression`` is used, it treats the supplied expression as
+a subiterator, producing values from it until the underlying iterator is
+exhausted.
+
+For full details of :keyword:`yield` semantics, refer to the :ref:`yieldexpr`
+section.
+
.. seealso::
:pep:`0255` - Simple Generators
The proposal for adding generators and the :keyword:`yield` statement to Python.
:pep:`0342` - Coroutines via Enhanced Generators
- The proposal that, among other generator enhancements, proposed allowing
- :keyword:`yield` to appear inside a :keyword:`try` ... :keyword:`finally` block.
+ The proposal to enhance the API and syntax of generators, making them
+ usable as simple coroutines.
+
+ :pep:`0380` - Syntax for Delegating to a Subgenerator
+ The proposal to introduce the :token:`yield_from` syntax, making delegation
+ to sub-generators easy.
.. _raise:
diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -195,6 +195,22 @@
print("You are not allowed to read document.txt")
+PEP 380: Syntax for Delegating to a Subgenerator
+================================================
+
+PEP 380 adds the ``yield from`` expression, allowing a generator to delegate
+part of its operations to another generator. This allows a section of code
+containing 'yield' to be factored out and placed in another generator.
+Additionally, the subgenerator is allowed to return with a value, and the
+value is made available to the delegating generator.
+While designed primarily for use in delegating to a subgenerator, the ``yield
+from`` expression actually allows delegation to arbitrary subiterators.
+
+(Implementation by Greg Ewing, integrated into 3.3 by Renaud Blanch, Ryan
+Kelly and Nick Coghlan, documentation by Zbigniew Jędrzejewski-Szmek and
+Nick Coghlan)
+
+
PEP 3155: Qualified name for classes and functions
==================================================
@@ -208,7 +224,6 @@
how they might be accessible from the global scope.
Example with (non-bound) methods::
-
>>> class C:
... def meth(self):
... pass
diff --git a/Grammar/Grammar b/Grammar/Grammar
--- a/Grammar/Grammar
+++ b/Grammar/Grammar
@@ -121,7 +121,7 @@
|'**' test)
# The reason that keywords are test nodes instead of NAME is that using NAME
# results in an ambiguity. ast.c makes sure it's a NAME.
-argument: test [comp_for] | test '=' test # Really [keyword '='] test
+argument: (test) [comp_for] | test '=' test # Really [keyword '='] test
comp_iter: comp_for | comp_if
comp_for: 'for' exprlist 'in' or_test [comp_iter]
comp_if: 'if' test_nocond [comp_iter]
@@ -129,4 +129,5 @@
# not used in grammar, but may appear in "node" passed from Parser to Compiler
encoding_decl: NAME
-yield_expr: 'yield' [testlist]
+yield_expr: 'yield' [yield_arg]
+yield_arg: 'from' test | testlist
diff --git a/Include/Python-ast.h b/Include/Python-ast.h
--- a/Include/Python-ast.h
+++ b/Include/Python-ast.h
@@ -245,6 +245,7 @@
} GeneratorExp;
struct {
+ int is_from;
expr_ty value;
} Yield;
@@ -487,8 +488,9 @@
#define GeneratorExp(a0, a1, a2, a3, a4) _Py_GeneratorExp(a0, a1, a2, a3, a4)
expr_ty _Py_GeneratorExp(expr_ty elt, asdl_seq * generators, int lineno, int
col_offset, PyArena *arena);
-#define Yield(a0, a1, a2, a3) _Py_Yield(a0, a1, a2, a3)
-expr_ty _Py_Yield(expr_ty value, int lineno, int col_offset, PyArena *arena);
+#define Yield(a0, a1, a2, a3, a4) _Py_Yield(a0, a1, a2, a3, a4)
+expr_ty _Py_Yield(int is_from, expr_ty value, int lineno, int col_offset,
+ PyArena *arena);
#define Compare(a0, a1, a2, a3, a4, a5) _Py_Compare(a0, a1, a2, a3, a4, a5)
expr_ty _Py_Compare(expr_ty left, asdl_int_seq * ops, asdl_seq * comparators,
int lineno, int col_offset, PyArena *arena);
diff --git a/Include/frameobject.h b/Include/frameobject.h
--- a/Include/frameobject.h
+++ b/Include/frameobject.h
@@ -9,45 +9,46 @@
#endif
typedef struct {
- int b_type; /* what kind of block this is */
- int b_handler; /* where to jump to find handler */
- int b_level; /* value stack level to pop to */
+ int b_type; /* what kind of block this is */
+ int b_handler; /* where to jump to find handler */
+ int b_level; /* value stack level to pop to */
} PyTryBlock;
typedef struct _frame {
PyObject_VAR_HEAD
- struct _frame *f_back; /* previous frame, or NULL */
- PyCodeObject *f_code; /* code segment */
- PyObject *f_builtins; /* builtin symbol table (PyDictObject) */
- PyObject *f_globals; /* global symbol table (PyDictObject) */
- PyObject *f_locals; /* local symbol table (any mapping) */
- PyObject **f_valuestack; /* points after the last local */
+ struct _frame *f_back; /* previous frame, or NULL */
+ PyCodeObject *f_code; /* code segment */
+ PyObject *f_builtins; /* builtin symbol table (PyDictObject) */
+ PyObject *f_globals; /* global symbol table (PyDictObject) */
+ PyObject *f_locals; /* local symbol table (any mapping) */
+ PyObject **f_valuestack; /* points after the last local */
/* Next free slot in f_valuestack. Frame creation sets to f_valuestack.
Frame evaluation usually NULLs it, but a frame that yields sets it
to the current stack top. */
PyObject **f_stacktop;
- PyObject *f_trace; /* Trace function */
+ PyObject *f_trace; /* Trace function */
+ PyObject *f_yieldfrom; /* Iterator being delegated to by yield from */
- /* In a generator, we need to be able to swap between the exception
- state inside the generator and the exception state of the calling
- frame (which shouldn't be impacted when the generator "yields"
- from an except handler).
- These three fields exist exactly for that, and are unused for
- non-generator frames. See the SAVE_EXC_STATE and SWAP_EXC_STATE
- macros in ceval.c for details of their use. */
+ /* In a generator, we need to be able to swap between the exception
+ state inside the generator and the exception state of the calling
+ frame (which shouldn't be impacted when the generator "yields"
+ from an except handler).
+ These three fields exist exactly for that, and are unused for
+ non-generator frames. See the SAVE_EXC_STATE and SWAP_EXC_STATE
+ macros in ceval.c for details of their use. */
PyObject *f_exc_type, *f_exc_value, *f_exc_traceback;
PyThreadState *f_tstate;
- int f_lasti; /* Last instruction if called */
+ int f_lasti; /* Last instruction if called */
/* Call PyFrame_GetLineNumber() instead of reading this field
directly. As of 2.3 f_lineno is only valid when tracing is
active (i.e. when f_trace is set). At other times we use
PyCode_Addr2Line to calculate the line from the current
bytecode index. */
- int f_lineno; /* Current line number */
- int f_iblock; /* index in f_blockstack */
+ int f_lineno; /* Current line number */
+ int f_iblock; /* index in f_blockstack */
PyTryBlock f_blockstack[CO_MAXBLOCKS]; /* for try and loop blocks */
- PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */
+ PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */
} PyFrameObject;
diff --git a/Include/genobject.h b/Include/genobject.h
--- a/Include/genobject.h
+++ b/Include/genobject.h
@@ -11,20 +11,20 @@
struct _frame; /* Avoid including frameobject.h */
typedef struct {
- PyObject_HEAD
- /* The gi_ prefix is intended to remind of generator-iterator. */
+ PyObject_HEAD
+ /* The gi_ prefix is intended to remind of generator-iterator. */
- /* Note: gi_frame can be NULL if the generator is "finished" */
- struct _frame *gi_frame;
+ /* Note: gi_frame can be NULL if the generator is "finished" */
+ struct _frame *gi_frame;
- /* True if generator is being executed. */
- int gi_running;
+ /* True if generator is being executed. */
+ int gi_running;
- /* The code object backing the generator */
- PyObject *gi_code;
+ /* The code object backing the generator */
+ PyObject *gi_code;
- /* List of weak reference. */
- PyObject *gi_weakreflist;
+ /* List of weak reference. */
+ PyObject *gi_weakreflist;
} PyGenObject;
PyAPI_DATA(PyTypeObject) PyGen_Type;
@@ -34,6 +34,7 @@
PyAPI_FUNC(PyObject *) PyGen_New(struct _frame *);
PyAPI_FUNC(int) PyGen_NeedsFinalizing(PyGenObject *);
+PyAPI_FUNC(int) PyGen_FetchStopIterationValue(PyObject **);
#ifdef __cplusplus
}
diff --git a/Include/graminit.h b/Include/graminit.h
--- a/Include/graminit.h
+++ b/Include/graminit.h
@@ -81,3 +81,4 @@
#define comp_if 334
#define encoding_decl 335
#define yield_expr 336
+#define yield_arg 337
diff --git a/Include/opcode.h b/Include/opcode.h
--- a/Include/opcode.h
+++ b/Include/opcode.h
@@ -7,116 +7,117 @@
/* Instruction opcodes for compiled code */
-#define POP_TOP 1
-#define ROT_TWO 2
-#define ROT_THREE 3
-#define DUP_TOP 4
+#define POP_TOP 1
+#define ROT_TWO 2
+#define ROT_THREE 3
+#define DUP_TOP 4
#define DUP_TOP_TWO 5
-#define NOP 9
+#define NOP 9
-#define UNARY_POSITIVE 10
-#define UNARY_NEGATIVE 11
-#define UNARY_NOT 12
+#define UNARY_POSITIVE 10
+#define UNARY_NEGATIVE 11
+#define UNARY_NOT 12
-#define UNARY_INVERT 15
+#define UNARY_INVERT 15
-#define BINARY_POWER 19
+#define BINARY_POWER 19
-#define BINARY_MULTIPLY 20
+#define BINARY_MULTIPLY 20
-#define BINARY_MODULO 22
-#define BINARY_ADD 23
-#define BINARY_SUBTRACT 24
-#define BINARY_SUBSCR 25
+#define BINARY_MODULO 22
+#define BINARY_ADD 23
+#define BINARY_SUBTRACT 24
+#define BINARY_SUBSCR 25
#define BINARY_FLOOR_DIVIDE 26
#define BINARY_TRUE_DIVIDE 27
#define INPLACE_FLOOR_DIVIDE 28
#define INPLACE_TRUE_DIVIDE 29
-#define STORE_MAP 54
-#define INPLACE_ADD 55
-#define INPLACE_SUBTRACT 56
-#define INPLACE_MULTIPLY 57
+#define STORE_MAP 54
+#define INPLACE_ADD 55
+#define INPLACE_SUBTRACT 56
+#define INPLACE_MULTIPLY 57
-#define INPLACE_MODULO 59
-#define STORE_SUBSCR 60
-#define DELETE_SUBSCR 61
+#define INPLACE_MODULO 59
+#define STORE_SUBSCR 60
+#define DELETE_SUBSCR 61
-#define BINARY_LSHIFT 62
-#define BINARY_RSHIFT 63
-#define BINARY_AND 64
-#define BINARY_XOR 65
-#define BINARY_OR 66
-#define INPLACE_POWER 67
-#define GET_ITER 68
-#define STORE_LOCALS 69
-#define PRINT_EXPR 70
+#define BINARY_LSHIFT 62
+#define BINARY_RSHIFT 63
+#define BINARY_AND 64
+#define BINARY_XOR 65
+#define BINARY_OR 66
+#define INPLACE_POWER 67
+#define GET_ITER 68
+#define STORE_LOCALS 69
+#define PRINT_EXPR 70
#define LOAD_BUILD_CLASS 71
+#define YIELD_FROM 72
-#define INPLACE_LSHIFT 75
-#define INPLACE_RSHIFT 76
-#define INPLACE_AND 77
-#define INPLACE_XOR 78
-#define INPLACE_OR 79
-#define BREAK_LOOP 80
+#define INPLACE_LSHIFT 75
+#define INPLACE_RSHIFT 76
+#define INPLACE_AND 77
+#define INPLACE_XOR 78
+#define INPLACE_OR 79
+#define BREAK_LOOP 80
#define WITH_CLEANUP 81
-#define RETURN_VALUE 83
-#define IMPORT_STAR 84
+#define RETURN_VALUE 83
+#define IMPORT_STAR 84
-#define YIELD_VALUE 86
-#define POP_BLOCK 87
-#define END_FINALLY 88
-#define POP_EXCEPT 89
+#define YIELD_VALUE 86
+#define POP_BLOCK 87
+#define END_FINALLY 88
+#define POP_EXCEPT 89
-#define HAVE_ARGUMENT 90 /* Opcodes from here have an argument: */
+#define HAVE_ARGUMENT 90 /* Opcodes from here have an argument: */
-#define STORE_NAME 90 /* Index in name list */
-#define DELETE_NAME 91 /* "" */
-#define UNPACK_SEQUENCE 92 /* Number of sequence items */
-#define FOR_ITER 93
+#define STORE_NAME 90 /* Index in name list */
+#define DELETE_NAME 91 /* "" */
+#define UNPACK_SEQUENCE 92 /* Number of sequence items */
+#define FOR_ITER 93
#define UNPACK_EX 94 /* Num items before variable part +
(Num items after variable part << 8) */
-#define STORE_ATTR 95 /* Index in name list */
-#define DELETE_ATTR 96 /* "" */
-#define STORE_GLOBAL 97 /* "" */
-#define DELETE_GLOBAL 98 /* "" */
+#define STORE_ATTR 95 /* Index in name list */
+#define DELETE_ATTR 96 /* "" */
+#define STORE_GLOBAL 97 /* "" */
+#define DELETE_GLOBAL 98 /* "" */
-#define LOAD_CONST 100 /* Index in const list */
-#define LOAD_NAME 101 /* Index in name list */
-#define BUILD_TUPLE 102 /* Number of tuple items */
-#define BUILD_LIST 103 /* Number of list items */
-#define BUILD_SET 104 /* Number of set items */
-#define BUILD_MAP 105 /* Always zero for now */
-#define LOAD_ATTR 106 /* Index in name list */
-#define COMPARE_OP 107 /* Comparison operator */
-#define IMPORT_NAME 108 /* Index in name list */
-#define IMPORT_FROM 109 /* Index in name list */
+#define LOAD_CONST 100 /* Index in const list */
+#define LOAD_NAME 101 /* Index in name list */
+#define BUILD_TUPLE 102 /* Number of tuple items */
+#define BUILD_LIST 103 /* Number of list items */
+#define BUILD_SET 104 /* Number of set items */
+#define BUILD_MAP 105 /* Always zero for now */
+#define LOAD_ATTR 106 /* Index in name list */
+#define COMPARE_OP 107 /* Comparison operator */
+#define IMPORT_NAME 108 /* Index in name list */
+#define IMPORT_FROM 109 /* Index in name list */
-#define JUMP_FORWARD 110 /* Number of bytes to skip */
-#define JUMP_IF_FALSE_OR_POP 111 /* Target byte offset from beginning of code */
-#define JUMP_IF_TRUE_OR_POP 112 /* "" */
-#define JUMP_ABSOLUTE 113 /* "" */
-#define POP_JUMP_IF_FALSE 114 /* "" */
-#define POP_JUMP_IF_TRUE 115 /* "" */
+#define JUMP_FORWARD 110 /* Number of bytes to skip */
+#define JUMP_IF_FALSE_OR_POP 111 /* Target byte offset from beginning of code */
+#define JUMP_IF_TRUE_OR_POP 112 /* "" */
+#define JUMP_ABSOLUTE 113 /* "" */
+#define POP_JUMP_IF_FALSE 114 /* "" */
+#define POP_JUMP_IF_TRUE 115 /* "" */
-#define LOAD_GLOBAL 116 /* Index in name list */
+#define LOAD_GLOBAL 116 /* Index in name list */
-#define CONTINUE_LOOP 119 /* Start of loop (absolute) */
-#define SETUP_LOOP 120 /* Target address (relative) */
-#define SETUP_EXCEPT 121 /* "" */
-#define SETUP_FINALLY 122 /* "" */
+#define CONTINUE_LOOP 119 /* Start of loop (absolute) */
+#define SETUP_LOOP 120 /* Target address (relative) */
+#define SETUP_EXCEPT 121 /* "" */
+#define SETUP_FINALLY 122 /* "" */
-#define LOAD_FAST 124 /* Local variable number */
-#define STORE_FAST 125 /* Local variable number */
-#define DELETE_FAST 126 /* Local variable number */
+#define LOAD_FAST 124 /* Local variable number */
+#define STORE_FAST 125 /* Local variable number */
+#define DELETE_FAST 126 /* Local variable number */
-#define RAISE_VARARGS 130 /* Number of raise arguments (1, 2 or 3) */
+#define RAISE_VARARGS 130 /* Number of raise arguments (1, 2 or 3) */
/* CALL_FUNCTION_XXX opcodes defined below depend on this definition */
-#define CALL_FUNCTION 131 /* #args + (#kwargs<<8) */
-#define MAKE_FUNCTION 132 /* #defaults + #kwdefaults<<8 + #annotations<<16 */
-#define BUILD_SLICE 133 /* Number of items */
+#define CALL_FUNCTION 131 /* #args + (#kwargs<<8) */
+#define MAKE_FUNCTION 132 /* #defaults + #kwdefaults<<8 + #annotations<<16 */
+#define BUILD_SLICE 133 /* Number of items */
#define MAKE_CLOSURE 134 /* same as MAKE_FUNCTION */
#define LOAD_CLOSURE 135 /* Load free variable from closure */
@@ -126,9 +127,9 @@
/* The next 3 opcodes must be contiguous and satisfy
(CALL_FUNCTION_VAR - CALL_FUNCTION) & 3 == 1 */
-#define CALL_FUNCTION_VAR 140 /* #args + (#kwargs<<8) */
-#define CALL_FUNCTION_KW 141 /* #args + (#kwargs<<8) */
-#define CALL_FUNCTION_VAR_KW 142 /* #args + (#kwargs<<8) */
+#define CALL_FUNCTION_VAR 140 /* #args + (#kwargs<<8) */
+#define CALL_FUNCTION_KW 141 /* #args + (#kwargs<<8) */
+#define CALL_FUNCTION_VAR_KW 142 /* #args + (#kwargs<<8) */
#define SETUP_WITH 143
@@ -148,7 +149,7 @@
enum cmp_op {PyCmp_LT=Py_LT, PyCmp_LE=Py_LE, PyCmp_EQ=Py_EQ, PyCmp_NE=Py_NE, PyCmp_GT=Py_GT, PyCmp_GE=Py_GE,
- PyCmp_IN, PyCmp_NOT_IN, PyCmp_IS, PyCmp_IS_NOT, PyCmp_EXC_MATCH, PyCmp_BAD};
+ PyCmp_IN, PyCmp_NOT_IN, PyCmp_IS, PyCmp_IS_NOT, PyCmp_EXC_MATCH, PyCmp_BAD};
#define HAS_ARG(op) ((op) >= HAVE_ARGUMENT)
diff --git a/Include/pyerrors.h b/Include/pyerrors.h
--- a/Include/pyerrors.h
+++ b/Include/pyerrors.h
@@ -51,6 +51,11 @@
Py_ssize_t written; /* only for BlockingIOError, -1 otherwise */
} PyOSErrorObject;
+typedef struct {
+ PyException_HEAD
+ PyObject *value;
+} PyStopIterationObject;
+
/* Compatibility typedefs */
typedef PyOSErrorObject PyEnvironmentErrorObject;
#ifdef MS_WINDOWS
@@ -380,6 +385,8 @@
const char *reason /* UTF-8 encoded string */
);
+/* create a StopIteration exception with the given value */
+PyAPI_FUNC(PyObject *) PyStopIteration_Create(PyObject *);
/* These APIs aren't really part of the error implementation, but
often needed to format error messages; the native C lib APIs are
diff --git a/Lib/opcode.py b/Lib/opcode.py
--- a/Lib/opcode.py
+++ b/Lib/opcode.py
@@ -87,6 +87,7 @@
def_op('PRINT_EXPR', 70)
def_op('LOAD_BUILD_CLASS', 71)
+def_op('YIELD_FROM', 72)
def_op('INPLACE_LSHIFT', 75)
def_op('INPLACE_RSHIFT', 76)
diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py
--- a/Lib/test/test_ast.py
+++ b/Lib/test/test_ast.py
@@ -813,7 +813,8 @@
self._check_comprehension(factory)
def test_yield(self):
- self.expr(ast.Yield(ast.Name("x", ast.Store())), "must have Load")
+ self.expr(ast.Yield(0, ast.Name("x", ast.Store())), "must have Load")
+ self.expr(ast.Yield(1, ast.Name("x", ast.Store())), "must have Load")
def test_compare(self):
left = ast.Name("x", ast.Load())
diff --git a/Lib/test/test_generators.py b/Lib/test/test_generators.py
--- a/Lib/test/test_generators.py
+++ b/Lib/test/test_generators.py
@@ -728,29 +728,6 @@
syntax_tests = """
->>> def f():
-... return 22
-... yield 1
-Traceback (most recent call last):
- ..
-SyntaxError: 'return' with argument inside generator
-
->>> def f():
-... yield 1
-... return 22
-Traceback (most recent call last):
- ..
-SyntaxError: 'return' with argument inside generator
-
-"return None" is not the same as "return" in a generator:
-
->>> def f():
-... yield 1
-... return None
-Traceback (most recent call last):
- ..
-SyntaxError: 'return' with argument inside generator
-
These are fine:
>>> def f():
@@ -866,20 +843,6 @@
>>> type(f())
<class 'generator'>
-
->>> def f():
-... if 0:
-... lambda x: x # shouldn't trigger here
-... return # or here
-... def f(i):
-... return 2*i # or here
-... if 0:
-... return 3 # but *this* sucks (line 8)
-... if 0:
-... yield 2 # because it's a generator (line 10)
-Traceback (most recent call last):
-SyntaxError: 'return' with argument inside generator
-
This one caused a crash (see SF bug 567538):
>>> def f():
@@ -1566,11 +1529,6 @@
...
SyntaxError: 'yield' outside function
->>> def f(): return lambda x=(yield): 1
-Traceback (most recent call last):
- ...
-SyntaxError: 'return' with argument inside generator
-
>>> def f(): x = yield = y
Traceback (most recent call last):
...
diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py
--- a/Lib/test/test_grammar.py
+++ b/Lib/test/test_grammar.py
@@ -458,7 +458,39 @@
check_syntax_error(self, "class foo:return 1")
def test_yield(self):
+ # Allowed as standalone statement
+ def g(): yield 1
+ def g(): yield from ()
+ # Allowed as RHS of assignment
+ def g(): x = yield 1
+ def g(): x = yield from ()
+ # Ordinary yield accepts implicit tuples
+ def g(): yield 1, 1
+ def g(): x = yield 1, 1
+ # 'yield from' does not
+ check_syntax_error(self, "def g(): yield from (), 1")
+ check_syntax_error(self, "def g(): x = yield from (), 1")
+ # Requires parentheses as subexpression
+ def g(): 1, (yield 1)
+ def g(): 1, (yield from ())
+ check_syntax_error(self, "def g(): 1, yield 1")
+ check_syntax_error(self, "def g(): 1, yield from ()")
+ # Requires parentheses as call argument
+ def g(): f((yield 1))
+ def g(): f((yield 1), 1)
+ def g(): f((yield from ()))
+ def g(): f((yield from ()), 1)
+ check_syntax_error(self, "def g(): f(yield 1)")
+ check_syntax_error(self, "def g(): f(yield 1, 1)")
+ check_syntax_error(self, "def g(): f(yield from ())")
+ check_syntax_error(self, "def g(): f(yield from (), 1)")
+ # Not allowed at top level
+ check_syntax_error(self, "yield")
+ check_syntax_error(self, "yield from")
+ # Not allowed at class scope
check_syntax_error(self, "class foo:yield 1")
+ check_syntax_error(self, "class foo:yield from ()")
+
def test_raise(self):
# 'raise' test [',' test]
diff --git a/Lib/test/test_parser.py b/Lib/test/test_parser.py
--- a/Lib/test/test_parser.py
+++ b/Lib/test/test_parser.py
@@ -50,6 +50,10 @@
self.check_suite("def f(): (yield 1)*2")
self.check_suite("def f(): return; yield 1")
self.check_suite("def f(): yield 1; return")
+ self.check_suite("def f(): yield from 1")
+ self.check_suite("def f(): x = yield from 1")
+ self.check_suite("def f(): f((yield from 1))")
+ self.check_suite("def f(): yield 1; return 1")
self.check_suite("def f():\n"
" for x in range(30):\n"
" yield x\n")
@@ -621,7 +625,6 @@
with self.assertRaises(TypeError):
parser.expr("a", "b")
-
def test_main():
support.run_unittest(
RoundtripLegalSyntaxTestCase,
diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py
--- a/Lib/test/test_sys.py
+++ b/Lib/test/test_sys.py
@@ -727,7 +727,7 @@
nfrees = len(x.f_code.co_freevars)
extras = x.f_code.co_stacksize + x.f_code.co_nlocals +\
ncells + nfrees - 1
- check(x, size(vh + '12P3i' + CO_MAXBLOCKS*'3i' + 'P' + extras*'P'))
+ check(x, size(vh + '13P3i' + CO_MAXBLOCKS*'3i' + 'P' + extras*'P'))
# function
def func(): pass
check(func, size(h + '12P'))
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -96,6 +96,7 @@
Philippe Biondi
Stuart Bishop
Roy Bixler
+Renaud Blanch
Mike Bland
Martin Bless
Pablo Bleyer
@@ -482,6 +483,7 @@
Jack Jansen
Bill Janssen
Thomas Jarosch
+Zbigniew Jędrzejewski-Szmek
Julien Jehannet
Drew Jenkins
Flemming Kjær Jensen
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
Core and Builtins
-----------------
+- PEP 380, Issue #11682: Add "yield from <x>" to support easy delegation to
+ subgenerators (initial patch by Greg Ewing, integration into 3.3 by
+ Renaud Blanch, Ryan Kelly, Zbigniew Jędrzejewski-Szmek and Nick Coghlan)
+
- Issue #13748: Raw bytes literals can now be written with the ``rb`` prefix
as well as ``br``.
diff --git a/Modules/parsermodule.c b/Modules/parsermodule.c
--- a/Modules/parsermodule.c
+++ b/Modules/parsermodule.c
@@ -298,25 +298,25 @@
/* Convert return value to a Boolean */
switch (op) {
- case Py_EQ:
+ case Py_EQ:
v = TEST_COND(result == 0);
break;
- case Py_NE:
+ case Py_NE:
v = TEST_COND(result != 0);
break;
- case Py_LE:
+ case Py_LE:
v = TEST_COND(result <= 0);
break;
- case Py_GE:
+ case Py_GE:
v = TEST_COND(result >= 0);
break;
- case Py_LT:
+ case Py_LT:
v = TEST_COND(result < 0);
break;
- case Py_GT:
+ case Py_GT:
v = TEST_COND(result > 0);
break;
- default:
+ default:
PyErr_BadArgument();
return NULL;
}
@@ -976,6 +976,7 @@
VALIDATER(testlist_comp); VALIDATER(yield_expr);
VALIDATER(or_test);
VALIDATER(test_nocond); VALIDATER(lambdef_nocond);
+VALIDATER(yield_arg);
#undef VALIDATER
@@ -1636,22 +1637,49 @@
}
-/* yield_expr: 'yield' [testlist]
+/* yield_expr: 'yield' [yield_arg]
*/
static int
validate_yield_expr(node *tree)
{
int nch = NCH(tree);
- int res = (validate_ntype(tree, yield_expr)
- && ((nch == 1) || (nch == 2))
- && validate_name(CHILD(tree, 0), "yield"));
-
- if (res && (nch == 2))
- res = validate_testlist(CHILD(tree, 1));
-
- return (res);
+ if (nch < 1 || nch > 2)
+ return 0;
+ if (!validate_ntype(tree, yield_expr))
+ return 0;
+ if (!validate_name(CHILD(tree, 0), "yield"))
+ return 0;
+ if (nch == 2) {
+ if (!validate_yield_arg(CHILD(tree, 1)))
+ return 0;
+ }
+ return 1;
}
+/* yield_arg: 'from' test | testlist
+ */
+static int
+validate_yield_arg(node *tree)
+{
+ int nch = NCH(tree);
+ if (!validate_ntype(tree, yield_arg))
+ return 0;
+ switch (nch) {
+ case 1:
+ if (!validate_testlist(CHILD(tree, nch - 1)))
+ return 0;
+ break;
+ case 2:
+ if (!validate_name(CHILD(tree, 0), "from"))
+ return 0;
+ if (!validate_test(CHILD(tree, 1)))
+ return 0;
+ break;
+ default:
+ return 0;
+ }
+ return 1;
+}
/* yield_stmt: yield_expr
*/
@@ -2120,16 +2148,16 @@
*/
tree = CHILD(tree, 0);
switch (TYPE(tree)) {
- case LESS:
- case GREATER:
- case EQEQUAL:
- case EQUAL:
- case LESSEQUAL:
- case GREATEREQUAL:
- case NOTEQUAL:
+ case LESS:
+ case GREATER:
+ case EQEQUAL:
+ case EQUAL:
+ case LESSEQUAL:
+ case GREATEREQUAL:
+ case NOTEQUAL:
res = 1;
break;
- case NAME:
+ case NAME:
res = ((strcmp(STR(tree), "in") == 0)
|| (strcmp(STR(tree), "is") == 0));
if (!res) {
@@ -2665,9 +2693,9 @@
{
int nch = NCH(tree);
int res = (validate_ntype(tree, argument)
- && ((nch == 1) || (nch == 2) || (nch == 3))
- && validate_test(CHILD(tree, 0)));
-
+ && ((nch == 1) || (nch == 2) || (nch == 3)));
+ if (res)
+ res = validate_test(CHILD(tree, 0));
if (res && (nch == 2))
res = validate_comp_for(CHILD(tree, 1));
else if (res && (nch == 3))
diff --git a/Objects/abstract.c b/Objects/abstract.c
--- a/Objects/abstract.c
+++ b/Objects/abstract.c
@@ -2267,7 +2267,6 @@
func = PyObject_GetAttrString(o, name);
if (func == NULL) {
- PyErr_SetString(PyExc_AttributeError, name);
return 0;
}
@@ -2311,7 +2310,6 @@
func = PyObject_GetAttrString(o, name);
if (func == NULL) {
- PyErr_SetString(PyExc_AttributeError, name);
return 0;
}
va_start(va, format);
diff --git a/Objects/exceptions.c b/Objects/exceptions.c
--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -487,8 +487,70 @@
/*
* StopIteration extends Exception
*/
-SimpleExtendsException(PyExc_Exception, StopIteration,
- "Signal the end from iterator.__next__().");
+
+static PyMemberDef StopIteration_members[] = {
+ {"value", T_OBJECT, offsetof(PyStopIterationObject, value), 0,
+ PyDoc_STR("generator return value")},
+ {NULL} /* Sentinel */
+};
+
+static int
+StopIteration_init(PyStopIterationObject *self, PyObject *args, PyObject *kwds)
+{
+ Py_ssize_t size = PyTuple_GET_SIZE(args);
+ PyObject *value;
+
+ if (BaseException_init((PyBaseExceptionObject *)self, args, kwds) == -1)
+ return -1;
+ Py_CLEAR(self->value);
+ if (size > 0)
+ value = PyTuple_GET_ITEM(args, 0);
+ else
+ value = Py_None;
+ Py_INCREF(value);
+ self->value = value;
+ return 0;
+}
+
+static int
+StopIteration_clear(PyStopIterationObject *self)
+{
+ Py_CLEAR(self->value);
+ return BaseException_clear((PyBaseExceptionObject *)self);
+}
+
+static void
+StopIteration_dealloc(PyStopIterationObject *self)
+{
+ _PyObject_GC_UNTRACK(self);
+ StopIteration_clear(self);
+ Py_TYPE(self)->tp_free((PyObject *)self);
+}
+
+static int
+StopIteration_traverse(PyStopIterationObject *self, visitproc visit, void *arg)
+{
+ Py_VISIT(self->value);
+ return BaseException_traverse((PyBaseExceptionObject *)self, visit, arg);
+}
+
+PyObject *
+PyStopIteration_Create(PyObject *value)
+{
+ return PyObject_CallFunctionObjArgs(PyExc_StopIteration, value, NULL);
+}
+
+ComplexExtendsException(
+ PyExc_Exception, /* base */
+ StopIteration, /* name */
+ StopIteration, /* prefix for *_init, etc */
+ 0, /* new */
+ 0, /* methods */
+ StopIteration_members, /* members */
+ 0, /* getset */
+ 0, /* str */
+ "Signal the end from iterator.__next__()."
+);
/*
diff --git a/Objects/frameobject.c b/Objects/frameobject.c
--- a/Objects/frameobject.c
+++ b/Objects/frameobject.c
@@ -15,11 +15,12 @@
#define OFF(x) offsetof(PyFrameObject, x)
static PyMemberDef frame_memberlist[] = {
- {"f_back", T_OBJECT, OFF(f_back), READONLY},
- {"f_code", T_OBJECT, OFF(f_code), READONLY},
- {"f_builtins", T_OBJECT, OFF(f_builtins),READONLY},
- {"f_globals", T_OBJECT, OFF(f_globals), READONLY},
- {"f_lasti", T_INT, OFF(f_lasti), READONLY},
+ {"f_back", T_OBJECT, OFF(f_back), READONLY},
+ {"f_code", T_OBJECT, OFF(f_code), READONLY},
+ {"f_builtins", T_OBJECT, OFF(f_builtins), READONLY},
+ {"f_globals", T_OBJECT, OFF(f_globals), READONLY},
+ {"f_lasti", T_INT, OFF(f_lasti), READONLY},
+ {"f_yieldfrom", T_OBJECT, OFF(f_yieldfrom), READONLY},
{NULL} /* Sentinel */
};
@@ -444,6 +445,7 @@
Py_CLEAR(f->f_exc_type);
Py_CLEAR(f->f_exc_value);
Py_CLEAR(f->f_exc_traceback);
+ Py_CLEAR(f->f_yieldfrom);
co = f->f_code;
if (co->co_zombieframe == NULL)
@@ -475,6 +477,7 @@
Py_VISIT(f->f_exc_type);
Py_VISIT(f->f_exc_value);
Py_VISIT(f->f_exc_traceback);
+ Py_VISIT(f->f_yieldfrom);
/* locals */
slots = f->f_code->co_nlocals + PyTuple_GET_SIZE(f->f_code->co_cellvars) + PyTuple_GET_SIZE(f->f_code->co_freevars);
@@ -508,6 +511,7 @@
Py_CLEAR(f->f_exc_value);
Py_CLEAR(f->f_exc_traceback);
Py_CLEAR(f->f_trace);
+ Py_CLEAR(f->f_yieldfrom);
/* locals */
slots = f->f_code->co_nlocals + PyTuple_GET_SIZE(f->f_code->co_cellvars) + PyTuple_GET_SIZE(f->f_code->co_freevars);
@@ -711,6 +715,7 @@
f->f_lasti = -1;
f->f_lineno = code->co_firstlineno;
f->f_iblock = 0;
+ f->f_yieldfrom = NULL;
_PyObject_GC_TRACK(f);
return f;
diff --git a/Objects/genobject.c b/Objects/genobject.c
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -5,6 +5,9 @@
#include "structmember.h"
#include "opcode.h"
+static PyObject *gen_close(PyGenObject *gen, PyObject *args);
+static void gen_undelegate(PyGenObject *gen);
+
static int
gen_traverse(PyGenObject *gen, visitproc visit, void *arg)
{
@@ -90,12 +93,18 @@
/* If the generator just returned (as opposed to yielding), signal
* that the generator is exhausted. */
- if (result == Py_None && f->f_stacktop == NULL) {
- Py_DECREF(result);
- result = NULL;
- /* Set exception if not called by gen_iternext() */
- if (arg)
+ if (result && f->f_stacktop == NULL) {
+ if (result == Py_None) {
+ /* Delay exception instantiation if we can */
PyErr_SetNone(PyExc_StopIteration);
+ } else {
+ PyObject *e = PyStopIteration_Create(result);
+ if (e != NULL) {
+ PyErr_SetObject(PyExc_StopIteration, e);
+ Py_DECREF(e);
+ }
+ }
+ Py_CLEAR(result);
}
if (!result || f->f_stacktop == NULL) {
@@ -111,8 +120,8 @@
Py_XDECREF(t);
Py_XDECREF(v);
Py_XDECREF(tb);
+ gen->gi_frame = NULL;
Py_DECREF(f);
- gen->gi_frame = NULL;
}
return result;
@@ -125,17 +134,91 @@
static PyObject *
gen_send(PyGenObject *gen, PyObject *arg)
{
- return gen_send_ex(gen, arg, 0);
+ int exc = 0;
+ PyObject *ret;
+ PyObject *yf = gen->gi_frame ? gen->gi_frame->f_yieldfrom : NULL;
+ /* XXX (ncoghlan): Are the incref/decref on arg and yf strictly needed?
+ * Or would it be valid to rely on borrowed references?
+ */
+ Py_INCREF(arg);
+ if (yf) {
+ Py_INCREF(yf);
+ if (PyGen_CheckExact(yf)) {
+ ret = gen_send((PyGenObject *)yf, arg);
+ } else {
+ if (arg == Py_None)
+ ret = PyIter_Next(yf);
+ else
+ ret = PyObject_CallMethod(yf, "send", "O", arg);
+ }
+ if (ret) {
+ Py_DECREF(yf);
+ goto done;
+ }
+ gen_undelegate(gen);
+ Py_CLEAR(arg);
+ if (PyGen_FetchStopIterationValue(&arg) < 0) {
+ exc = 1;
+ }
+ Py_DECREF(yf);
+ }
+ ret = gen_send_ex(gen, arg, exc);
+done:
+ Py_XDECREF(arg);
+ return ret;
}
PyDoc_STRVAR(close_doc,
"close(arg) -> raise GeneratorExit inside generator.");
+/*
+ * This helper function is used by gen_close and gen_throw to
+ * close a subiterator being delegated to by yield-from.
+ */
+
+static int
+gen_close_iter(PyObject *yf)
+{
+ PyObject *retval = NULL;
+
+ if (PyGen_CheckExact(yf)) {
+ retval = gen_close((PyGenObject *)yf, NULL);
+ if (retval == NULL) {
+ return -1;
+ }
+ } else {
+ PyObject *meth = PyObject_GetAttrString(yf, "close");
+ if (meth == NULL) {
+ if (!PyErr_ExceptionMatches(PyExc_AttributeError)) {
+ PyErr_WriteUnraisable(yf);
+ }
+ PyErr_Clear();
+ } else {
+ retval = PyObject_CallFunction(meth, "");
+ Py_DECREF(meth);
+ if (!retval)
+ return -1;
+ }
+ }
+ Py_XDECREF(retval);
+ return 0;
+}
+
static PyObject *
gen_close(PyGenObject *gen, PyObject *args)
{
PyObject *retval;
- PyErr_SetNone(PyExc_GeneratorExit);
+ PyObject *yf = gen->gi_frame ? gen->gi_frame->f_yieldfrom : NULL;
+ int err = 0;
+
+ if (yf) {
+ Py_INCREF(yf);
+ err = gen_close_iter(yf);
+ gen_undelegate(gen);
+ Py_DECREF(yf);
+ }
+ if (err == 0)
+ PyErr_SetNone(PyExc_GeneratorExit);
retval = gen_send_ex(gen, Py_None, 1);
if (retval) {
Py_DECREF(retval);
@@ -196,7 +279,7 @@
_Py_NewReference(self);
self->ob_refcnt = refcnt;
}
- assert(PyType_IS_GC(self->ob_type) &&
+ assert(PyType_IS_GC(Py_TYPE(self)) &&
_Py_AS_GC(self)->gc.gc_refs != _PyGC_REFS_UNTRACKED);
/* If Py_REF_DEBUG, _Py_NewReference bumped _Py_RefTotal, so
@@ -209,8 +292,8 @@
* undone.
*/
#ifdef COUNT_ALLOCS
- --self->ob_type->tp_frees;
- --self->ob_type->tp_allocs;
+ --(Py_TYPE(self)->tp_frees);
+ --(Py_TYPE(self)->tp_allocs);
#endif
}
@@ -226,10 +309,55 @@
PyObject *typ;
PyObject *tb = NULL;
PyObject *val = NULL;
+ PyObject *yf = gen->gi_frame ? gen->gi_frame->f_yieldfrom : NULL;
if (!PyArg_UnpackTuple(args, "throw", 1, 3, &typ, &val, &tb))
return NULL;
+ if (yf) {
+ PyObject *ret;
+ int err;
+ Py_INCREF(yf);
+ if (PyErr_GivenExceptionMatches(typ, PyExc_GeneratorExit)) {
+ err = gen_close_iter(yf);
+ Py_DECREF(yf);
+ gen_undelegate(gen);
+ if (err < 0)
+ return gen_send_ex(gen, Py_None, 1);
+ goto throw_here;
+ }
+ if (PyGen_CheckExact(yf)) {
+ ret = gen_throw((PyGenObject *)yf, args);
+ } else {
+ PyObject *meth = PyObject_GetAttrString(yf, "throw");
+ if (meth == NULL) {
+ if (!PyErr_ExceptionMatches(PyExc_AttributeError)) {
+ Py_DECREF(yf);
+ return NULL;
+ }
+ PyErr_Clear();
+ Py_DECREF(yf);
+ gen_undelegate(gen);
+ goto throw_here;
+ }
+ ret = PyObject_CallObject(meth, args);
+ Py_DECREF(meth);
+ }
+ Py_DECREF(yf);
+ if (!ret) {
+ PyObject *val;
+ gen_undelegate(gen);
+ if (PyGen_FetchStopIterationValue(&val) == 0) {
+ ret = gen_send_ex(gen, val, 0);
+ Py_DECREF(val);
+ } else {
+ ret = gen_send_ex(gen, Py_None, 1);
+ }
+ }
+ return ret;
+ }
+
+throw_here:
/* First, check the traceback argument, replacing None with
NULL. */
if (tb == Py_None) {
@@ -272,7 +400,7 @@
PyErr_Format(PyExc_TypeError,
"exceptions must be classes or instances "
"deriving from BaseException, not %s",
- typ->ob_type->tp_name);
+ Py_TYPE(typ)->tp_name);
goto failed_throw;
}
@@ -291,9 +419,74 @@
static PyObject *
gen_iternext(PyGenObject *gen)
{
- return gen_send_ex(gen, NULL, 0);
+ PyObject *val = NULL;
+ PyObject *ret;
+ int exc = 0;
+ PyObject *yf = gen->gi_frame ? gen->gi_frame->f_yieldfrom : NULL;
+ if (yf) {
+ Py_INCREF(yf);
+ /* ceval.c ensures that yf is an iterator */
+ ret = Py_TYPE(yf)->tp_iternext(yf);
+ if (ret) {
+ Py_DECREF(yf);
+ return ret;
+ }
+ gen_undelegate(gen);
+ if (PyGen_FetchStopIterationValue(&val) < 0)
+ exc = 1;
+ Py_DECREF(yf);
+ }
+ ret = gen_send_ex(gen, val, exc);
+ Py_XDECREF(val);
+ return ret;
}
+/*
+ * In certain recursive situations, a generator may lose its frame
+ * before we get a chance to clear f_yieldfrom, so we use this
+ * helper function.
+ */
+
+static void
+gen_undelegate(PyGenObject *gen) {
+ if (gen->gi_frame) {
+ Py_XDECREF(gen->gi_frame->f_yieldfrom);
+ gen->gi_frame->f_yieldfrom = NULL;
+ }
+}
+
+/*
+ * If StopIteration exception is set, fetches its 'value'
+ * attribute if any, otherwise sets pvalue to None.
+ *
+ * Returns 0 if no exception or StopIteration is set.
+ * If any other exception is set, returns -1 and leaves
+ * pvalue unchanged.
+ */
+
+int
+PyGen_FetchStopIterationValue(PyObject **pvalue) {
+ PyObject *et, *ev, *tb;
+ PyObject *value = NULL;
+
+ if (PyErr_ExceptionMatches(PyExc_StopIteration)) {
+ PyErr_Fetch(&et, &ev, &tb);
+ Py_XDECREF(et);
+ Py_XDECREF(tb);
+ if (ev) {
+ value = ((PyStopIterationObject *)ev)->value;
+ Py_DECREF(ev);
+ }
+ } else if (PyErr_Occurred()) {
+ return -1;
+ }
+ if (value == NULL) {
+ value = Py_None;
+ }
+ Py_INCREF(value);
+ *pvalue = value;
+ return 0;
+}
static PyObject *
gen_repr(PyGenObject *gen)
diff --git a/Parser/Python.asdl b/Parser/Python.asdl
--- a/Parser/Python.asdl
+++ b/Parser/Python.asdl
@@ -59,7 +59,7 @@
| DictComp(expr key, expr value, comprehension* generators)
| GeneratorExp(expr elt, comprehension* generators)
-- the grammar constrains where yield expressions can occur
- | Yield(expr? value)
+ | Yield(int is_from, expr? value)
-- need sequences for compare to distinguish between
-- x < 4 < 3 and (x < 4) < 3
| Compare(expr left, cmpop* ops, expr* comparators)
diff --git a/Python/Python-ast.c b/Python/Python-ast.c
--- a/Python/Python-ast.c
+++ b/Python/Python-ast.c
@@ -231,7 +231,9 @@
"generators",
};
static PyTypeObject *Yield_type;
+_Py_IDENTIFIER(is_from);
static char *Yield_fields[]={
+ "is_from",
"value",
};
static PyTypeObject *Compare_type;
@@ -810,7 +812,7 @@
GeneratorExp_type = make_type("GeneratorExp", expr_type,
GeneratorExp_fields, 2);
if (!GeneratorExp_type) return 0;
- Yield_type = make_type("Yield", expr_type, Yield_fields, 1);
+ Yield_type = make_type("Yield", expr_type, Yield_fields, 2);
if (!Yield_type) return 0;
Compare_type = make_type("Compare", expr_type, Compare_fields, 3);
if (!Compare_type) return 0;
@@ -1747,13 +1749,14 @@
}
expr_ty
-Yield(expr_ty value, int lineno, int col_offset, PyArena *arena)
+Yield(int is_from, expr_ty value, int lineno, int col_offset, PyArena *arena)
{
expr_ty p;
p = (expr_ty)PyArena_Malloc(arena, sizeof(*p));
if (!p)
return NULL;
p->kind = Yield_kind;
+ p->v.Yield.is_from = is_from;
p->v.Yield.value = value;
p->lineno = lineno;
p->col_offset = col_offset;
@@ -2795,6 +2798,11 @@
case Yield_kind:
result = PyType_GenericNew(Yield_type, NULL, NULL);
if (!result) goto failed;
+ value = ast2obj_int(o->v.Yield.is_from);
+ if (!value) goto failed;
+ if (PyObject_SetAttrString(result, "is_from", value) == -1)
+ goto failed;
+ Py_DECREF(value);
value = ast2obj_expr(o->v.Yield.value);
if (!value) goto failed;
if (_PyObject_SetAttrId(result, &PyId_value, value) == -1)
@@ -5337,8 +5345,21 @@
return 1;
}
if (isinstance) {
+ int is_from;
expr_ty value;
+ if (_PyObject_HasAttrId(obj, &PyId_is_from)) {
+ int res;
+ tmp = _PyObject_GetAttrId(obj, &PyId_is_from);
+ if (tmp == NULL) goto failed;
+ res = obj2ast_int(tmp, &is_from, arena);
+ if (res != 0) goto failed;
+ Py_XDECREF(tmp);
+ tmp = NULL;
+ } else {
+ PyErr_SetString(PyExc_TypeError, "required field \"is_from\" missing from Yield");
+ return 1;
+ }
if (_PyObject_HasAttrId(obj, &PyId_value)) {
int res;
tmp = _PyObject_GetAttrId(obj, &PyId_value);
@@ -5350,7 +5371,7 @@
} else {
value = NULL;
}
- *out = Yield(value, lineno, col_offset, arena);
+ *out = Yield(is_from, value, lineno, col_offset, arena);
if (*out == NULL) goto failed;
return 0;
}
diff --git a/Python/ast.c b/Python/ast.c
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -2369,13 +2369,24 @@
}
return ast_for_binop(c, n);
case yield_expr: {
+ node *an = NULL;
+ node *en = NULL;
+ int is_from = 0;
expr_ty exp = NULL;
- if (NCH(n) == 2) {
- exp = ast_for_testlist(c, CHILD(n, 1));
+ if (NCH(n) > 1)
+ an = CHILD(n, 1); /* yield_arg */
+ if (an) {
+ en = CHILD(an, NCH(an) - 1);
+ if (NCH(an) == 2) {
+ is_from = 1;
+ exp = ast_for_expr(c, en);
+ }
+ else
+ exp = ast_for_testlist(c, en);
if (!exp)
return NULL;
}
- return Yield(exp, LINENO(n), n->n_col_offset, c->c_arena);
+ return Yield(is_from, exp, LINENO(n), n->n_col_offset, c->c_arena);
}
case factor:
if (NCH(n) == 1) {
@@ -2399,7 +2410,7 @@
/*
arglist: (argument ',')* (argument [',']| '*' test [',' '**' test]
| '**' test)
- argument: [test '='] test [comp_for] # Really [keyword '='] test
+ argument: [test '='] (test) [comp_for] # Really [keyword '='] test
*/
int i, nargs, nkeywords, ngens;
@@ -2693,7 +2704,7 @@
continue_stmt: 'continue'
return_stmt: 'return' [testlist]
yield_stmt: yield_expr
- yield_expr: 'yield' testlist
+ yield_expr: 'yield' testlist | 'yield' 'from' test
raise_stmt: 'raise' [test [',' test [',' test]]]
*/
node *ch;
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1828,6 +1828,52 @@
why = WHY_RETURN;
goto fast_block_end;
+ TARGET(YIELD_FROM)
+ u = POP();
+ x = PyObject_GetIter(u);
+ Py_DECREF(u);
+ if (x == NULL)
+ break;
+ /* x is now the iterator, make the first next() call */
+ retval = (*Py_TYPE(x)->tp_iternext)(x);
+ if (!retval) {
+ /* iter may be exhausted */
+ Py_CLEAR(x);
+ if (!PyErr_ExceptionMatches(PyExc_StopIteration)) {
+ /* some other exception */
+ break;
+ }
+ /* try to get return value from exception */
+ PyObject *et, *ev, *tb;
+ PyErr_Fetch(&et, &ev, &tb);
+ Py_XDECREF(et);
+ Py_XDECREF(tb);
+ /* u is return value */
+ u = NULL;
+ if (ev) {
+ u = PyObject_GetAttrString(ev, "value");
+ Py_DECREF(ev);
+ if (u == NULL) {
+ if (!PyErr_ExceptionMatches(PyExc_AttributeError)) {
+ /* some other exception */
+ break;
+ }
+ PyErr_Clear();
+ }
+ }
+ if (u == NULL) {
+ u = Py_None;
+ Py_INCREF(u);
+ }
+ PUSH(u);
+ continue;
+ }
+ /* x is iterator, retval is value to be yielded */
+ f->f_yieldfrom = x;
+ f->f_stacktop = stack_pointer;
+ why = WHY_YIELD;
+ goto fast_yield;
+
TARGET(YIELD_VALUE)
retval = POP();
f->f_stacktop = stack_pointer;
diff --git a/Python/compile.c b/Python/compile.c
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -840,6 +840,7 @@
case IMPORT_STAR:
return -1;
case YIELD_VALUE:
+ case YIELD_FROM:
return 0;
case POP_BLOCK:
@@ -3318,7 +3319,12 @@
else {
ADDOP_O(c, LOAD_CONST, Py_None, consts);
}
- ADDOP(c, YIELD_VALUE);
+ if (e->v.Yield.is_from) {
+ ADDOP(c, YIELD_FROM);
+ }
+ else {
+ ADDOP(c, YIELD_VALUE);
+ }
break;
case Compare_kind:
return compiler_compare(c, e);
diff --git a/Python/graminit.c b/Python/graminit.c
--- a/Python/graminit.c
+++ b/Python/graminit.c
@@ -1791,7 +1791,7 @@
{167, 1},
};
static arc arcs_80_1[2] = {
- {9, 2},
+ {168, 2},
{0, 1},
};
static arc arcs_80_2[1] = {
@@ -1802,171 +1802,188 @@
{2, arcs_80_1},
{1, arcs_80_2},
};
-static dfa dfas[81] = {
+static arc arcs_81_0[2] = {
+ {73, 1},
+ {9, 2},
+};
+static arc arcs_81_1[1] = {
+ {24, 2},
+};
+static arc arcs_81_2[1] = {
+ {0, 2},
+};
+static state states_81[3] = {
+ {2, arcs_81_0},
+ {1, arcs_81_1},
+ {1, arcs_81_2},
+};
+static dfa dfas[82] = {
{256, "single_input", 0, 3, states_0,
- "\004\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204"},
+ "\004\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204\000"},
{257, "file_input", 0, 2, states_1,
- "\204\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204"},
+ "\204\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204\000"},
{258, "eval_input", 0, 3, states_2,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{259, "decorator", 0, 7, states_3,
- "\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{260, "decorators", 0, 2, states_4,
- "\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{261, "decorated", 0, 3, states_5,
- "\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{262, "funcdef", 0, 8, states_6,
- "\000\000\020\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\020\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{263, "parameters", 0, 4, states_7,
- "\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{264, "typedargslist", 0, 18, states_8,
- "\000\000\040\200\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\200\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{265, "tfpdef", 0, 4, states_9,
- "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{266, "varargslist", 0, 18, states_10,
- "\000\000\040\200\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\200\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{267, "vfpdef", 0, 2, states_11,
- "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{268, "stmt", 0, 2, states_12,
- "\000\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204"},
+ "\000\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204\000"},
{269, "simple_stmt", 0, 4, states_13,
- "\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200"},
+ "\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200\000"},
{270, "small_stmt", 0, 2, states_14,
- "\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200"},
+ "\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200\000"},
{271, "expr_stmt", 0, 6, states_15,
- "\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{272, "testlist_star_expr", 0, 3, states_16,
- "\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{273, "augassign", 0, 2, states_17,
- "\000\000\000\000\000\000\376\037\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\376\037\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{274, "del_stmt", 0, 3, states_18,
- "\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{275, "pass_stmt", 0, 2, states_19,
- "\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{276, "flow_stmt", 0, 2, states_20,
- "\000\000\000\000\000\000\000\000\340\001\000\000\000\000\000\000\000\000\000\000\200"},
+ "\000\000\000\000\000\000\000\000\340\001\000\000\000\000\000\000\000\000\000\000\200\000"},
{277, "break_stmt", 0, 2, states_21,
- "\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{278, "continue_stmt", 0, 2, states_22,
- "\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{279, "return_stmt", 0, 3, states_23,
- "\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{280, "yield_stmt", 0, 2, states_24,
- "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000"},
{281, "raise_stmt", 0, 5, states_25,
- "\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000"},
{282, "import_stmt", 0, 2, states_26,
- "\000\000\000\000\000\000\000\000\000\022\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\022\000\000\000\000\000\000\000\000\000\000\000\000"},
{283, "import_name", 0, 3, states_27,
- "\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000\000\000"},
{284, "import_from", 0, 8, states_28,
- "\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000"},
{285, "import_as_name", 0, 4, states_29,
- "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{286, "dotted_as_name", 0, 4, states_30,
- "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{287, "import_as_names", 0, 3, states_31,
- "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{288, "dotted_as_names", 0, 2, states_32,
- "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{289, "dotted_name", 0, 2, states_33,
- "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{290, "global_stmt", 0, 3, states_34,
- "\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000\000"},
{291, "nonlocal_stmt", 0, 3, states_35,
- "\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000"},
{292, "assert_stmt", 0, 5, states_36,
- "\000\000\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000"},
{293, "compound_stmt", 0, 2, states_37,
- "\000\010\020\000\000\000\000\000\000\000\000\220\045\000\000\000\000\000\000\000\004"},
+ "\000\010\020\000\000\000\000\000\000\000\000\220\045\000\000\000\000\000\000\000\004\000"},
{294, "if_stmt", 0, 8, states_38,
- "\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000"},
{295, "while_stmt", 0, 8, states_39,
- "\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000"},
{296, "for_stmt", 0, 10, states_40,
- "\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000"},
{297, "try_stmt", 0, 13, states_41,
- "\000\000\000\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\000"},
{298, "with_stmt", 0, 5, states_42,
- "\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000"},
{299, "with_item", 0, 4, states_43,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{300, "except_clause", 0, 5, states_44,
- "\000\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000"},
{301, "suite", 0, 5, states_45,
- "\004\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200"},
+ "\004\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200\000"},
{302, "test", 0, 6, states_46,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{303, "test_nocond", 0, 2, states_47,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{304, "lambdef", 0, 5, states_48,
- "\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000"},
{305, "lambdef_nocond", 0, 5, states_49,
- "\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000"},
{306, "or_test", 0, 2, states_50,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000\000"},
{307, "and_test", 0, 2, states_51,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000\000"},
{308, "not_test", 0, 3, states_52,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000\000"},
{309, "comparison", 0, 2, states_53,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{310, "comp_op", 0, 4, states_54,
- "\000\000\000\000\000\000\000\000\000\000\000\000\002\000\220\177\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\002\000\220\177\000\000\000\000\000\000"},
{311, "star_expr", 0, 3, states_55,
- "\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{312, "expr", 0, 2, states_56,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{313, "xor_expr", 0, 2, states_57,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{314, "and_expr", 0, 2, states_58,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{315, "shift_expr", 0, 2, states_59,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{316, "arith_expr", 0, 2, states_60,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{317, "term", 0, 2, states_61,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{318, "factor", 0, 3, states_62,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{319, "power", 0, 4, states_63,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000\000"},
{320, "atom", 0, 9, states_64,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000\000"},
{321, "testlist_comp", 0, 5, states_65,
- "\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{322, "trailer", 0, 7, states_66,
- "\000\040\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\020\000\000"},
+ "\000\040\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\020\000\000\000"},
{323, "subscriptlist", 0, 3, states_67,
- "\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{324, "subscript", 0, 5, states_68,
- "\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{325, "sliceop", 0, 3, states_69,
- "\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{326, "exprlist", 0, 3, states_70,
- "\000\040\040\200\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
+ "\000\040\040\200\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{327, "testlist", 0, 3, states_71,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{328, "dictorsetmaker", 0, 11, states_72,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{329, "classdef", 0, 8, states_73,
- "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\004"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\004\000"},
{330, "arglist", 0, 8, states_74,
- "\000\040\040\200\001\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\200\001\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{331, "argument", 0, 4, states_75,
- "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
+ "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{332, "comp_iter", 0, 2, states_76,
- "\000\000\000\000\000\000\000\000\000\000\000\020\001\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\020\001\000\000\000\000\000\000\000\000\000"},
{333, "comp_for", 0, 6, states_77,
- "\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000"},
{334, "comp_if", 0, 4, states_78,
- "\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000"},
+ "\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000"},
{335, "encoding_decl", 0, 2, states_79,
- "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
+ "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{336, "yield_expr", 0, 3, states_80,
- "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200"},
+ "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000"},
+ {337, "yield_arg", 0, 3, states_81,
+ "\000\040\040\000\000\000\000\000\000\202\000\000\000\200\020\000\000\206\120\076\000\000"},
};
-static label labels[168] = {
+static label labels[169] = {
{0, "EMPTY"},
{256, 0},
{4, 0},
@@ -2135,10 +2152,11 @@
{334, 0},
{335, 0},
{1, "yield"},
+ {337, 0},
};
grammar _PyParser_Grammar = {
- 81,
+ 82,
dfas,
- {168, labels},
+ {169, labels},
256
};
diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h
--- a/Python/opcode_targets.h
+++ b/Python/opcode_targets.h
@@ -71,7 +71,7 @@
&&TARGET_STORE_LOCALS,
&&TARGET_PRINT_EXPR,
&&TARGET_LOAD_BUILD_CLASS,
- &&_unknown_opcode,
+ &&TARGET_YIELD_FROM,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_INPLACE_LSHIFT,
diff --git a/Python/symtable.c b/Python/symtable.c
--- a/Python/symtable.c
+++ b/Python/symtable.c
@@ -19,10 +19,6 @@
#define IMPORT_STAR_WARNING "import * only allowed at module level"
-#define RETURN_VAL_IN_GENERATOR \
- "'return' with argument inside generator"
-
-
static PySTEntryObject *
ste_new(struct symtable *st, identifier name, _Py_block_ty block,
void *key, int lineno, int col_offset)
@@ -1133,14 +1129,6 @@
if (s->v.Return.value) {
VISIT(st, expr, s->v.Return.value);
st->st_cur->ste_returns_value = 1;
- if (st->st_cur->ste_generator) {
- PyErr_SetString(PyExc_SyntaxError,
- RETURN_VAL_IN_GENERATOR);
- PyErr_SyntaxLocationEx(st->st_filename,
- s->lineno,
- s->col_offset);
- return 0;
- }
}
break;
case Delete_kind:
@@ -1345,13 +1333,6 @@
if (e->v.Yield.value)
VISIT(st, expr, e->v.Yield.value);
st->st_cur->ste_generator = 1;
- if (st->st_cur->ste_returns_value) {
- PyErr_SetString(PyExc_SyntaxError,
- RETURN_VAL_IN_GENERATOR);
- PyErr_SyntaxLocationEx(st->st_filename,
- e->lineno, e->col_offset);
- return 0;
- }
break;
case Compare_kind:
VISIT(st, expr, e->v.Compare.left);
--
Repository URL: http://hg.python.org/cpython
More information about the Python-checkins
mailing list