[Python-ideas] Changes to the existing optimization levels
Diana Clarke
diana.joan.clarke at gmail.com
Thu Sep 28 14:48:15 EDT 2017
Hi folks:
I was recently looking for an entry-level cpython task to work on in
my spare time and plucked this off of someone's TODO list.
"Make optimizations more fine-grained than just -O and -OO"
There are currently three supported optimization levels (0, 1, and 2).
Briefly summarized, they do the following.
0: no optimizations
1: remove assert statements and __debug__ blocks
2: remove docstrings, assert statements, and __debug__ blocks
>From what I gather, their use-case is assert statements in production
code. More specifically, they want to be able to optimize away
docstrings, but keep the assert statements, which currently isn't
possible with the existing optimization levels.
As a first baby-step, I considered just adding a new optimization
level 3 that keeps asserts but continues to remove docstrings and
__debug__ blocks.
3: remove docstrings and __debug__ blocks
>From a command-line perspective, there is already support for
additional optimization levels. That is, without making any changes,
the optimization level will increase with the number of 0s provided.
$ python -c "import sys; print(sys.flags.optimize)"
0
$ python -OO -c "import sys; print(sys.flags.optimize)"
2
$ python -OOOOOOO -c "import sys; print(sys.flags.optimize)"
7
And the PYTHONOPTIMIZE environment variable will happily assign
something like 42 to sys.flags.optimize.
$ unset PYTHONOPTIMIZE
$ python -c "import sys; print(sys.flags.optimize)"
0
$ export PYTHONOPTIMIZE=2
$ python -c "import sys; print(sys.flags.optimize)"
2
$ export PYTHONOPTIMIZE=42
$ python -c "import sys; print(sys.flags.optimize)"
42
Finally, the resulting __pycache__ folder also already contains the
expected bytecode files for the new optimization levels (
__init__.cpython-37.opt-42.pyc was created for optimization level 42,
for example).
$ tree
.
└── test
├── __init__.py
└── __pycache__
├── __init__.cpython-37.opt-1.pyc
├── __init__.cpython-37.opt-2.pyc
├── __init__.cpython-37.opt-42.pyc
├── __init__.cpython-37.opt-7.pyc
└── __init__.cpython-37.pyc
Adding optimization level 3 is an easy change to make. Here's that
quick proof of concept (minus changes to the docs, etc). I've also
attached that diff as 3.diff.
https://github.com/dianaclarke/cpython/commit/4bd7278d87bd762b2989178e5bfed309cf9fb5bf
I was initially looking for a more elegant solution that allowed you
to specify exactly which optimizations you wanted, and when I floated
this naive ("level 3") approach off-list to a few core developers,
their feedback confirmed my hunch (too hacky).
So for my second pass at this task, I started with the following two
pronged approach.
1) Changed the various compile signatures to accept a set of
string optimization flags rather than an int value.
2) Added a new command line option N that allows you to specify
any number of individual optimization flags.
For example:
python -N nodebug -N noassert -N nodocstring
The existing optimization options (-O and -OO) still exist in this
approach, but they are mapped to the new optimization flags
("nodebug", "noassert", "nodocstring").
With the exception of the builtin complile() function, all underlying
compile functions would only accept optimization flags going forward,
and the builtin compile() function would accept both an integer
optimize value or a set of optimization flags for backwards
compatibility.
You can find that work-in-progress approach here on github (also
attached as N.diff).
https://github.com/dianaclarke/cpython/commit/3e36cea1fc8ee6f4cdc584851e4c1edfc2bb1e56
All in all, that approach is going fairly well, but there's a lot of
work remaining, and that diff is already getting quite large (for my
new-contributor status).
Note for example, that I haven't yet tackled adding bytecode files to
__pycache__ that reflect these new optimization flags. Something like:
$ tree
.
└── test
├── __init__.py
└── __pycache__
├── __init__.cpython-37.opt-nodebug-noassert.pyc
├── __init__.cpython-37.opt-nodebug-nodocstring.pyc
├── __init__.cpython-37.opt-nodebug-noassert-nodocstring.pyc
└── __init__.cpython-37.pyc
I'm also not certain if the various compile signatures are even open
for change (int optimize => PyObject *optimizations), or if that's a
no-no.
And there are still a ton of references to "-O", "-OO",
"sys.flags.optimize", "Py_OptimizeFlag", "PYTHONOPTIMIZE", "optimize",
etc that all need to be audited and their implications considered.
I've really enjoyed this task and I'm learning a lot about the c api,
but I think this is a good place to stop and solicit feedback and
direction.
My gut says that the amount of churn and resulting risk is too high to
continue down this path, but I would love to hear thoughts from others
(alternate approaches, ways to limit scope, confirmation that the
existing approach is too entrenched for change, etc).
Regardless, I think the following subset change could merge without
any bigger picture changes, as it just adds test coverage for a case
not yet covered. I can reopen that pull request once I clean up the
commit message a bit (I closed it in the mean time).
https://github.com/python/cpython/pull/3450/commits/bfdab955a94a7fef431548f3ba2c4b5ca79e958d
Thanks for your time!
Cheers,
--diana
-------------- next part --------------
diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py
index 9d949b74cb..cc6baff707 100644
--- a/Lib/test/test_builtin.py
+++ b/Lib/test/test_builtin.py
@@ -339,7 +339,8 @@ class BuiltinTest(unittest.TestCase):
values = [(-1, __debug__, f.__doc__),
(0, True, 'doc'),
(1, False, 'doc'),
- (2, False, None)]
+ (2, False, None),
+ (3, True, None)]
for optval, debugval, docstring in values:
# test both direct compilation and compilation via AST
codeobjs = []
diff --git a/Modules/main.c b/Modules/main.c
index 08b22760de..a63882685e 100644
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -66,6 +66,7 @@ static const char usage_2[] = "\
-m mod : run library module as a script (terminates option list)\n\
-O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x\n\
-OO : remove doc-strings in addition to the -O optimizations\n\
+-OOO : like -OO but don't optimize away asserts\n\
-q : don't print version and copyright messages on interactive startup\n\
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE\n\
-S : don't imply 'import site' on initialization\n\
diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c
index 85f207b68e..453b9bbc8a 100644
--- a/Python/bltinmodule.c
+++ b/Python/bltinmodule.c
@@ -705,7 +705,7 @@ builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
}
/* XXX Warn if (supplied_flags & PyCF_MASK_OBSOLETE) != 0? */
- if (optimize < -1 || optimize > 2) {
+ if (optimize < -1 || optimize > 3) {
PyErr_SetString(PyExc_ValueError,
"compile(): invalid optimize value");
goto error;
diff --git a/Python/compile.c b/Python/compile.c
index 280ddc39e3..e892e34e6c 100644
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -1431,7 +1431,7 @@ compiler_body(struct compiler *c, asdl_seq *stmts, string docstring)
if (find_ann(stmts)) {
ADDOP(c, SETUP_ANNOTATIONS);
}
- /* if not -OO mode, set docstring */
+ /* if not -OO or -OOO mode, set docstring */
if (c->c_optimize < 2 && docstring) {
ADDOP_O(c, LOAD_CONST, docstring, consts);
ADDOP_NAME(c, STORE_NAME, __doc__, names);
@@ -1836,7 +1836,7 @@ compiler_function(struct compiler *c, stmt_ty s, int is_async)
return 0;
}
- /* if not -OO mode, add docstring */
+ /* if not -OO or -OOO mode, add docstring */
if (c->c_optimize < 2 && s->v.FunctionDef.docstring)
docstring = s->v.FunctionDef.docstring;
if (compiler_add_o(c, c->u->u_consts, docstring) < 0) {
@@ -2825,7 +2825,7 @@ compiler_assert(struct compiler *c, stmt_ty s)
basicblock *end;
PyObject* msg;
- if (c->c_optimize)
+ if (c->c_optimize && c->c_optimize != 3)
return 1;
if (assertion_error == NULL) {
assertion_error = PyUnicode_InternFromString("AssertionError");
-------------- next part --------------
diff --git a/Include/compile.h b/Include/compile.h
index 3cc351d409..837e4afbea 100644
--- a/Include/compile.h
+++ b/Include/compile.h
@@ -47,18 +47,18 @@ typedef struct {
#define FUTURE_GENERATOR_STOP "generator_stop"
struct _mod; /* Declare the existence of this type */
-#define PyAST_Compile(mod, s, f, ar) PyAST_CompileEx(mod, s, f, -1, ar)
+#define PyAST_Compile(mod, s, f, ar) PyAST_CompileEx(mod, s, f, NULL, ar)
PyAPI_FUNC(PyCodeObject *) PyAST_CompileEx(
struct _mod *mod,
const char *filename, /* decoded from the filesystem encoding */
PyCompilerFlags *flags,
- int optimize,
+ PyObject *optimizations,
PyArena *arena);
PyAPI_FUNC(PyCodeObject *) PyAST_CompileObject(
struct _mod *mod,
PyObject *filename,
PyCompilerFlags *flags,
- int optimize,
+ PyObject *optimizations,
PyArena *arena);
PyAPI_FUNC(PyFutureFeatures *) PyFuture_FromAST(
struct _mod * mod,
diff --git a/Include/pythonrun.h b/Include/pythonrun.h
index 6f0c6fc655..31625d002c 100644
--- a/Include/pythonrun.h
+++ b/Include/pythonrun.h
@@ -100,19 +100,19 @@ PyAPI_FUNC(PyObject *) PyRun_FileExFlags(
#ifdef Py_LIMITED_API
PyAPI_FUNC(PyObject *) Py_CompileString(const char *, const char *, int);
#else
-#define Py_CompileString(str, p, s) Py_CompileStringExFlags(str, p, s, NULL, -1)
-#define Py_CompileStringFlags(str, p, s, f) Py_CompileStringExFlags(str, p, s, f, -1)
+#define Py_CompileString(str, p, s) Py_CompileStringExFlags(str, p, s, NULL, NULL)
+#define Py_CompileStringFlags(str, p, s, f) Py_CompileStringExFlags(str, p, s, f, NULL)
PyAPI_FUNC(PyObject *) Py_CompileStringExFlags(
const char *str,
const char *filename, /* decoded from the filesystem encoding */
int start,
PyCompilerFlags *flags,
- int optimize);
+ PyObject *optimizations);
PyAPI_FUNC(PyObject *) Py_CompileStringObject(
const char *str,
PyObject *filename, int start,
PyCompilerFlags *flags,
- int optimize);
+ PyObject *optimizations);
#endif
PyAPI_FUNC(struct symtable *) Py_SymtableString(
const char *str,
diff --git a/Include/sysmodule.h b/Include/sysmodule.h
index c5547ff674..62bba3bd94 100644
--- a/Include/sysmodule.h
+++ b/Include/sysmodule.h
@@ -25,6 +25,9 @@ PyAPI_FUNC(void) PySys_WriteStderr(const char *format, ...)
PyAPI_FUNC(void) PySys_FormatStdout(const char *format, ...);
PyAPI_FUNC(void) PySys_FormatStderr(const char *format, ...);
+PyAPI_FUNC(void) PySys_SetOptimizations(PyObject *);
+PyAPI_FUNC(PyObject *) PySys_GetOptimizations(void);
+
PyAPI_FUNC(void) PySys_ResetWarnOptions(void);
PyAPI_FUNC(void) PySys_AddWarnOption(const wchar_t *);
PyAPI_FUNC(void) PySys_AddWarnOptionUnicode(PyObject *);
diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py
index 9d949b74cb..87dcda7b43 100644
--- a/Lib/test/test_builtin.py
+++ b/Lib/test/test_builtin.py
@@ -328,19 +328,22 @@ class BuiltinTest(unittest.TestCase):
codestr = '''def f():
"""doc"""
+ debug_enabled = False
+ if __debug__:
+ debug_enabled = True
try:
assert False
except AssertionError:
- return (True, f.__doc__)
+ return (True, f.__doc__, debug_enabled)
else:
- return (False, f.__doc__)
+ return (False, f.__doc__, debug_enabled)
'''
def f(): """doc"""
- values = [(-1, __debug__, f.__doc__),
- (0, True, 'doc'),
- (1, False, 'doc'),
- (2, False, None)]
- for optval, debugval, docstring in values:
+ values = [(-1, __debug__, f.__doc__, __debug__),
+ (0, True, 'doc', True),
+ (1, False, 'doc', False),
+ (2, False, None, False)]
+ for optval, assertval, docstring, debugval in values:
# test both direct compilation and compilation via AST
codeobjs = []
codeobjs.append(compile(codestr, "<test>", "exec", optimize=optval))
@@ -350,7 +353,7 @@ class BuiltinTest(unittest.TestCase):
ns = {}
exec(code, ns)
rv = ns['f']()
- self.assertEqual(rv, (debugval, docstring))
+ self.assertEqual(rv, (assertval, docstring, debugval))
def test_delattr(self):
sys.spam = 1
diff --git a/Modules/main.c b/Modules/main.c
index 08b22760de..051d9e4792 100644
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -40,7 +40,7 @@ static wchar_t **orig_argv;
static int orig_argc;
/* command line options */
-#define BASE_OPTS L"bBc:dEhiIJm:OqRsStuvVW:xX:?"
+#define BASE_OPTS L"bBc:dEhiIJm:N:OqRsStuvVW:xX:?"
#define PROGRAM_OPTS BASE_OPTS
@@ -64,6 +64,7 @@ static const char usage_2[] = "\
if stdin does not appear to be a terminal; also PYTHONINSPECT=x\n\
-I : isolate Python from the user's environment (implies -E and -s)\n\
-m mod : run library module as a script (terminates option list)\n\
+-N : optimization flags: nodebug, noassert, nodocstring\n\
-O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x\n\
-OO : remove doc-strings in addition to the -O optimizations\n\
-q : don't print version and copyright messages on interactive startup\n\
@@ -354,6 +355,7 @@ typedef struct {
wchar_t *module; /* -m argument */
PyObject *warning_options; /* -W options */
PyObject *extra_options; /* -X options */
+ PyObject *optimizations; /* -N optimization flags */
int print_help; /* -h, -? options */
int print_version; /* -V option */
int bytes_warning; /* Py_BytesWarningFlag */
@@ -372,7 +374,7 @@ typedef struct {
} _Py_CommandLineDetails;
#define _Py_CommandLineDetails_INIT \
- {NULL, NULL, NULL, NULL, NULL, \
+ {NULL, NULL, NULL, NULL, NULL, NULL, \
0, 0, 0, 0, 0, 0, 0, 0, \
0, 0, 0, 0, 0, 0, 0}
@@ -380,6 +382,7 @@ static int
read_command_line(int argc, wchar_t **argv, _Py_CommandLineDetails *cmdline)
{
PyObject *warning_option = NULL;
+ PyObject *optimization = NULL;
wchar_t *command = NULL;
wchar_t *module = NULL;
int c;
@@ -435,6 +438,14 @@ read_command_line(int argc, wchar_t **argv, _Py_CommandLineDetails *cmdline)
/* case 'J': reserved for Jython */
+ case 'N':
+ if (cmdline->optimizations == NULL)
+ cmdline->optimizations = PySet_New(NULL);
+ optimization = PyUnicode_FromWideChar(_PyOS_optarg, -1);
+ PySet_Add(cmdline->optimizations, optimization);
+ Py_DECREF(optimization);
+ break;
+
case 'O':
cmdline->optimization_level++;
break;
@@ -515,6 +526,46 @@ read_command_line(int argc, wchar_t **argv, _Py_CommandLineDetails *cmdline)
}
}
+ /*
+ * Map legacy optimization_level int value to optimization flags.
+ * 0: {}, no optimization flags
+ * 1: {"nodebug", "noassert"}
+ * 2: {"nodebug", "noassert", "nodocstring"}
+ */
+ if (cmdline->optimization_level >= 0) {
+ if (cmdline->optimizations == NULL) {
+ cmdline->optimizations = PySet_New(NULL);
+ }
+
+ if (cmdline->optimization_level == 1) {
+ optimization = PyUnicode_FromString("nodebug");
+ PySet_Add(cmdline->optimizations, optimization);
+ Py_DECREF(optimization);
+
+ optimization = PyUnicode_FromString("noassert");
+ PySet_Add(cmdline->optimizations, optimization);
+ Py_DECREF(optimization);
+ }
+
+ if (cmdline->optimization_level == 2) {
+ optimization = PyUnicode_FromString("nodebug");
+ PySet_Add(cmdline->optimizations, optimization);
+ Py_DECREF(optimization);
+
+ optimization = PyUnicode_FromString("noassert");
+ PySet_Add(cmdline->optimizations, optimization);
+ Py_DECREF(optimization);
+
+ optimization = PyUnicode_FromString("nodocstring");
+ PySet_Add(cmdline->optimizations, optimization);
+ Py_DECREF(optimization);
+ }
+ }
+
+ if (cmdline->optimizations != NULL) {
+ PySys_SetOptimizations(cmdline->optimizations);
+ }
+
if (command == NULL && module == NULL && _PyOS_optind < argc &&
wcscmp(argv[_PyOS_optind], L"-") != 0)
{
diff --git a/Modules/parsermodule.c b/Modules/parsermodule.c
index 929f2deb16..91e0286a6b 100644
--- a/Modules/parsermodule.c
+++ b/Modules/parsermodule.c
@@ -515,7 +515,7 @@ parser_compilest(PyST_Object *self, PyObject *args, PyObject *kw)
goto error;
res = (PyObject *)PyAST_CompileObject(mod, filename,
- &self->st_flags, -1, arena);
+ &self->st_flags, NULL, arena);
error:
Py_XDECREF(filename);
if (arena != NULL)
diff --git a/Modules/zipimport.c b/Modules/zipimport.c
index fad1b1f5ab..302a8b8bd3 100644
--- a/Modules/zipimport.c
+++ b/Modules/zipimport.c
@@ -1365,7 +1365,7 @@ compile_source(PyObject *pathname, PyObject *source)
}
code = Py_CompileStringObject(PyBytes_AsString(fixed_source),
- pathname, Py_file_input, NULL, -1);
+ pathname, Py_file_input, NULL, NULL);
Py_DECREF(fixed_source);
return code;
diff --git a/Programs/_freeze_importlib.c b/Programs/_freeze_importlib.c
index 1069966a18..7b9be6cfd1 100644
--- a/Programs/_freeze_importlib.c
+++ b/Programs/_freeze_importlib.c
@@ -90,7 +90,12 @@ main(int argc, char *argv[])
code_name = is_bootstrap ?
"<frozen importlib._bootstrap>" :
"<frozen importlib._bootstrap_external>";
- code = Py_CompileStringExFlags(text, code_name, Py_file_input, NULL, 0);
+
+ PyObject *optimizations = PySet_New(NULL); // empty: no optimizations
+ code = Py_CompileStringExFlags(text, code_name, Py_file_input, NULL,
+ optimizations);
+ Py_DECREF(optimizations);
+
if (code == NULL)
goto error;
free(text);
diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c
index 85f207b68e..542ae337e2 100644
--- a/Python/bltinmodule.c
+++ b/Python/bltinmodule.c
@@ -683,7 +683,7 @@ in addition to any features explicitly specified.
static PyObject *
builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
const char *mode, int flags, int dont_inherit,
- int optimize)
+ PyObject *optimizations)
/*[clinic end generated code: output=1fa176e33452bb63 input=0ff726f595eb9fcd]*/
{
PyObject *source_copy;
@@ -705,11 +705,11 @@ builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
}
/* XXX Warn if (supplied_flags & PyCF_MASK_OBSOLETE) != 0? */
- if (optimize < -1 || optimize > 2) {
- PyErr_SetString(PyExc_ValueError,
- "compile(): invalid optimize value");
- goto error;
- }
+// if (optimize < -1 || optimize > 2) {
+// PyErr_SetString(PyExc_ValueError,
+// "compile(): invalid optimize value");
+// goto error;
+// }
if (!dont_inherit) {
PyEval_MergeCompilerFlags(&cf);
@@ -751,8 +751,8 @@ builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
PyArena_Free(arena);
goto error;
}
- result = (PyObject*)PyAST_CompileObject(mod, filename,
- &cf, optimize, arena);
+ result = (PyObject*)PyAST_CompileObject(mod, filename, &cf,
+ optimizations, arena);
PyArena_Free(arena);
}
goto finally;
@@ -762,7 +762,8 @@ builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
if (str == NULL)
goto error;
- result = Py_CompileStringObject(str, filename, start[compile_mode], &cf, optimize);
+ result = Py_CompileStringObject(str, filename, start[compile_mode], &cf,
+ optimizations);
Py_XDECREF(source_copy);
goto finally;
@@ -2737,7 +2738,7 @@ _PyBuiltin_Init(void)
SETBUILTIN("tuple", &PyTuple_Type);
SETBUILTIN("type", &PyType_Type);
SETBUILTIN("zip", &PyZip_Type);
- debug = PyBool_FromLong(Py_OptimizeFlag == 0);
+ debug = PyBool_FromLong(Py_OptimizeFlag == 0); // TODO
if (PyDict_SetItemString(dict, "__debug__", debug) < 0) {
Py_DECREF(debug);
return NULL;
diff --git a/Python/clinic/bltinmodule.c.h b/Python/clinic/bltinmodule.c.h
index fa327da0ff..35accc0701 100644
--- a/Python/clinic/bltinmodule.c.h
+++ b/Python/clinic/bltinmodule.c.h
@@ -133,7 +133,7 @@ exit:
PyDoc_STRVAR(builtin_compile__doc__,
"compile($module, /, source, filename, mode, flags=0,\n"
-" dont_inherit=False, optimize=-1)\n"
+" dont_inherit=False, optimizations=None, optimize=-1)\n"
"--\n"
"\n"
"Compile source into a code object that can be executed by exec() or eval().\n"
@@ -155,26 +155,61 @@ PyDoc_STRVAR(builtin_compile__doc__,
static PyObject *
builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
const char *mode, int flags, int dont_inherit,
- int optimize);
+ PyObject *optimizations);
static PyObject *
builtin_compile(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames)
{
PyObject *return_value = NULL;
- static const char * const _keywords[] = {"source", "filename", "mode", "flags", "dont_inherit", "optimize", NULL};
- static _PyArg_Parser _parser = {"OO&s|iii:compile", _keywords, 0};
+ static const char * const _keywords[] = {
+ "source",
+ "filename",
+ "mode",
+ "flags",
+ "dont_inherit",
+ "optimizations",
+ "optimize",
+ NULL
+ };
+ static _PyArg_Parser _parser = {"OO&s|iiOi:compile", _keywords, 0};
PyObject *source;
PyObject *filename;
const char *mode;
int flags = 0;
int dont_inherit = 0;
+ PyObject *optimizations = NULL;
int optimize = -1;
if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser,
- &source, PyUnicode_FSDecoder, &filename, &mode, &flags, &dont_inherit, &optimize)) {
- goto exit;
+ &source, PyUnicode_FSDecoder, &filename, &mode, &flags, &dont_inherit,
+ &optimizations, &optimize)) {
+ goto exit;
+ }
+
+ /*
+ * Map legacy optimize int value to optimization flags.
+ * - 1: NULL, use optimization level of interpreter
+ * 0: {}, no optimization flags
+ * 1: {"nodebug", "noassert"}
+ * 2: {"nodebug", "noassert", "nodocstring"}
+ */
+ if (optimize >= 0) {
+ if (optimizations == NULL) {
+ optimizations = PySet_New(NULL);
+ }
+
+ if (optimize == 1) {
+ PySet_Add(optimizations, PyUnicode_FromString("nodebug"));
+ PySet_Add(optimizations, PyUnicode_FromString("noassert"));
+ } else if (optimize == 2) {
+ PySet_Add(optimizations, PyUnicode_FromString("nodebug"));
+ PySet_Add(optimizations, PyUnicode_FromString("noassert"));
+ PySet_Add(optimizations, PyUnicode_FromString("nodocstring"));
+ }
}
- return_value = builtin_compile_impl(module, source, filename, mode, flags, dont_inherit, optimize);
+
+ return_value = builtin_compile_impl(module, source, filename, mode, flags,
+ dont_inherit, optimizations);
exit:
return return_value;
diff --git a/Python/compile.c b/Python/compile.c
index 280ddc39e3..a062f3e0ee 100644
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -154,7 +154,7 @@ struct compiler {
PyFutureFeatures *c_future; /* pointer to module's __future__ */
PyCompilerFlags *c_flags;
- int c_optimize; /* optimization level */
+ PyObject *c_optimizations; /* optimization flags */
int c_interactive; /* true if in interactive mode */
int c_nestlevel;
@@ -285,6 +285,12 @@ _Py_Mangle(PyObject *privateobj, PyObject *ident)
return result;
}
+static int
+optimization_flag_absent(struct compiler *c, const char *flag)
+{
+ return PySet_Contains(c->c_optimizations, PyUnicode_FromString(flag)) == 0;
+}
+
static int
compiler_init(struct compiler *c)
{
@@ -299,7 +305,7 @@ compiler_init(struct compiler *c)
PyCodeObject *
PyAST_CompileObject(mod_ty mod, PyObject *filename, PyCompilerFlags *flags,
- int optimize, PyArena *arena)
+ PyObject *optimizations, PyArena *arena)
{
struct compiler c;
PyCodeObject *co = NULL;
@@ -328,9 +334,14 @@ PyAST_CompileObject(mod_ty mod, PyObject *filename, PyCompilerFlags *flags,
c.c_future->ff_features = merged;
flags->cf_flags = merged;
c.c_flags = flags;
- c.c_optimize = (optimize == -1) ? Py_OptimizeFlag : optimize;
c.c_nestlevel = 0;
+ if (optimizations == NULL) {
+ c.c_optimizations = PySys_GetOptimizations();
+ } else {
+ c.c_optimizations = optimizations;
+ }
+
c.c_st = PySymtable_BuildObject(mod, filename, c.c_future);
if (c.c_st == NULL) {
if (!PyErr_Occurred())
@@ -348,14 +359,14 @@ PyAST_CompileObject(mod_ty mod, PyObject *filename, PyCompilerFlags *flags,
PyCodeObject *
PyAST_CompileEx(mod_ty mod, const char *filename_str, PyCompilerFlags *flags,
- int optimize, PyArena *arena)
+ PyObject *optimizations, PyArena *arena)
{
PyObject *filename;
PyCodeObject *co;
filename = PyUnicode_DecodeFSDefault(filename_str);
if (filename == NULL)
return NULL;
- co = PyAST_CompileObject(mod, filename, flags, optimize, arena);
+ co = PyAST_CompileObject(mod, filename, flags, optimizations, arena);
Py_DECREF(filename);
return co;
@@ -1432,7 +1443,8 @@ compiler_body(struct compiler *c, asdl_seq *stmts, string docstring)
ADDOP(c, SETUP_ANNOTATIONS);
}
/* if not -OO mode, set docstring */
- if (c->c_optimize < 2 && docstring) {
+ int include_doc = optimization_flag_absent(c, "nodocstring");
+ if (include_doc && docstring) {
ADDOP_O(c, LOAD_CONST, docstring, consts);
ADDOP_NAME(c, STORE_NAME, __doc__, names);
}
@@ -1837,7 +1849,8 @@ compiler_function(struct compiler *c, stmt_ty s, int is_async)
}
/* if not -OO mode, add docstring */
- if (c->c_optimize < 2 && s->v.FunctionDef.docstring)
+ int include_doc = optimization_flag_absent(c, "nodocstring");
+ if (include_doc && s->v.FunctionDef.docstring)
docstring = s->v.FunctionDef.docstring;
if (compiler_add_o(c, c->u->u_consts, docstring) < 0) {
compiler_exit_scope(c);
@@ -2825,8 +2838,10 @@ compiler_assert(struct compiler *c, stmt_ty s)
basicblock *end;
PyObject* msg;
- if (c->c_optimize)
+ int include_assert = optimization_flag_absent(c, "noassert");
+ if (!include_assert) {
return 1;
+ }
if (assertion_error == NULL) {
assertion_error = PyUnicode_InternFromString("AssertionError");
if (assertion_error == NULL)
@@ -4142,8 +4157,10 @@ expr_constant(struct compiler *c, expr_ty e)
case Name_kind:
/* optimize away names that can't be reassigned */
id = PyUnicode_AsUTF8(e->v.Name.id);
- if (id && strcmp(id, "__debug__") == 0)
- return !c->c_optimize;
+ if (id && strcmp(id, "__debug__") == 0) {
+ int include_debug = optimization_flag_absent(c, "nodebug");
+ return include_debug ? 1 : 0;
+ }
return -1;
case NameConstant_kind: {
PyObject *o = e->v.NameConstant.value;
@@ -5453,5 +5470,5 @@ PyAPI_FUNC(PyCodeObject *)
PyAST_Compile(mod_ty mod, const char *filename, PyCompilerFlags *flags,
PyArena *arena)
{
- return PyAST_CompileEx(mod, filename, flags, -1, arena);
+ return PyAST_CompileEx(mod, filename, flags, NULL, arena);
}
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
index f31b3ee5a5..0225a9c382 100644
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -976,7 +976,7 @@ run_mod(mod_ty mod, PyObject *filename, PyObject *globals, PyObject *locals,
{
PyCodeObject *co;
PyObject *v;
- co = PyAST_CompileObject(mod, filename, flags, -1, arena);
+ co = PyAST_CompileObject(mod, filename, flags, NULL, arena);
if (co == NULL)
return NULL;
v = PyEval_EvalCode((PyObject*)co, globals, locals);
@@ -1023,7 +1023,7 @@ run_pyc_file(FILE *fp, const char *filename, PyObject *globals,
PyObject *
Py_CompileStringObject(const char *str, PyObject *filename, int start,
- PyCompilerFlags *flags, int optimize)
+ PyCompilerFlags *flags, PyObject *optimizations)
{
PyCodeObject *co;
mod_ty mod;
@@ -1041,20 +1041,20 @@ Py_CompileStringObject(const char *str, PyObject *filename, int start,
PyArena_Free(arena);
return result;
}
- co = PyAST_CompileObject(mod, filename, flags, optimize, arena);
+ co = PyAST_CompileObject(mod, filename, flags, optimizations, arena);
PyArena_Free(arena);
return (PyObject *)co;
}
PyObject *
Py_CompileStringExFlags(const char *str, const char *filename_str, int start,
- PyCompilerFlags *flags, int optimize)
+ PyCompilerFlags *flags, PyObject *optimizations)
{
PyObject *filename, *co;
filename = PyUnicode_DecodeFSDefault(filename_str);
if (filename == NULL)
return NULL;
- co = Py_CompileStringObject(str, filename, start, flags, optimize);
+ co = Py_CompileStringObject(str, filename, start, flags, optimizations);
Py_DECREF(filename);
return co;
}
@@ -1523,7 +1523,7 @@ PyRun_SimpleString(const char *s)
PyAPI_FUNC(PyObject *)
Py_CompileString(const char *str, const char *p, int s)
{
- return Py_CompileStringExFlags(str, p, s, NULL, -1);
+ return Py_CompileStringExFlags(str, p, s, NULL, NULL);
}
#undef Py_CompileStringFlags
@@ -1531,7 +1531,7 @@ PyAPI_FUNC(PyObject *)
Py_CompileStringFlags(const char *str, const char *p, int s,
PyCompilerFlags *flags)
{
- return Py_CompileStringExFlags(str, p, s, flags, -1);
+ return Py_CompileStringExFlags(str, p, s, flags, NULL);
}
#undef PyRun_InteractiveOne
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
index ab435c8310..2e325ad031 100644
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -1481,6 +1481,31 @@ list_builtin_module_names(void)
return list;
}
+static PyObject *optimizations = NULL;
+
+void
+PySys_SetOptimizations(PyObject *options)
+{
+ if (optimizations == NULL)
+ optimizations = PySet_New(NULL);
+
+ Py_ssize_t i;
+ int size = PySet_GET_SIZE(options);
+ for (i = 0; i < size; i++) {
+ PySet_Add(optimizations, PySet_Pop(options));
+ }
+}
+
+PyObject *
+PySys_GetOptimizations(void)
+{
+ if (optimizations == NULL || !PySet_Check(optimizations)) {
+ Py_XDECREF(optimizations);
+ optimizations = PySet_New(NULL);
+ }
+ return optimizations;
+}
+
static PyObject *warnoptions = NULL;
void
More information about the Python-ideas
mailing list