Macros instead of inline functions?
As I wander around the code base, I keep seeing macro definitions in the C code. For example, there are four CALL* macros defined in Python/ast_opt.c which contain not entirely trivial bits of syntax. That code is from 2017 (as compared to, say, Modules/audioop.c, which first saw the light of day in 1992) I see the inline keyword used unconditionally in many places. I don't think stable code which uses macros should be changed (though I see the INCREF/DECREF macros just call private inline functions, so some conversion has clearly been done). Still, in new code, shouldn't the use of macros for more than trivial use cases (constant defs, simple one-liners) be discouraged at this point? Skip
04.12.19 14:08, Skip Montanaro пише:
As I wander around the code base, I keep seeing macro definitions in the C code. For example, there are four CALL* macros defined in Python/ast_opt.c which contain not entirely trivial bits of syntax. That code is from 2017 (as compared to, say, Modules/audioop.c, which first saw the light of day in 1992) I see the inline keyword used unconditionally in many places.
I don't think stable code which uses macros should be changed (though I see the INCREF/DECREF macros just call private inline functions, so some conversion has clearly been done). Still, in new code, shouldn't the use of macros for more than trivial use cases (constant defs, simple one-liners) be discouraged at this point?
You can't goto from the inline function.
I don't think stable code which uses macros should be changed (though I see the INCREF/DECREF macros just call private inline functions, so some conversion has clearly been done). Still, in new code, shouldn't the use of macros for more than trivial use cases (constant defs, simple one-liners) be discouraged at this point?
You can't goto from the inline function.
Thanks, that is true, and if needed would add another case where macros are preferred over inline functions (as would use of the cpp token pasting operator, ##). I see relatively few goto-using macros spread across a number of source files, but the examples I highlighted in Python/ast_opt.c use return, not goto. It seems they could easily be crafted as inline functions which return 0 (forcing early return from enclosing function) or 1 (equivalent to current fall through). Still, I'm not terribly worried about existing usage, especially in stable, well-tested code. I guess I'm more wondering if a preference for inline functions shouldn't be mentioned in PEP 7 for future authors. Skip Skip
04.12.19 18:54, Skip Montanaro пише:
I don't think stable code which uses macros should be changed (though I see the INCREF/DECREF macros just call private inline functions, so some conversion has clearly been done). Still, in new code, shouldn't the use of macros for more than trivial use cases (constant defs, simple one-liners) be discouraged at this point?
You can't goto from the inline function.
Thanks, that is true, and if needed would add another case where macros are preferred over inline functions (as would use of the cpp token pasting operator, ##). I see relatively few goto-using macros spread across a number of source files, but the examples I highlighted in Python/ast_opt.c use return, not goto. It seems they could easily be crafted as inline functions which return 0 (forcing early return from enclosing function) or 1 (equivalent to current fall through).
In this case it is the same as a goto. Both affect the control flow of the caller. In these files (symtable.c, compile.c, ast_opt.c, etc) there are sequences of calls for child nodes. Every call can return an error, so you need to check every call and return an error immediately after the call. With inline functions you would need to write if (!VISIT(...)) { return 0; } if (!VISIT(...)) { return 0; } if (!VISIT(...)) { return 0; } instead of just VISIT(...); VISIT(...); VISIT(...); This will increase the number of lines by two or three (according to the current PEP 7 recommendations) times. This will add a lot of boilerplate which distracts the attention from the logic. It will need to pass all arguments explicitly (macros allow to pass some arguments implicitly). This will add a lot of spots for introducing errors. If you will need to change some details, for example decrement state->recursion_depth before return you will need to change handreds lines instead of few sites (and guess how much bugs you will introduce). Macros exist to get rid of the boilerplate. They are used not only in private files, this idiom is used in the public API. See the Py_VISIT macro. It passes implicit arguments and returns from the caller.
This is my last post on this, at least as far as specific usage instances are concerned. See my question about PEP 7 below. If that is a discussion people think worthwhile, please start a new thread.
if (!VISIT(...)) { return 0; } if (!VISIT(...)) { return 0; } if (!VISIT(...)) { return 0; }
instead of just
VISIT(...); VISIT(...); VISIT(...);
That seems easily solved with the VISIT-as-macro calling _VISIT-as-inline function. That pattern exists elsewhere in the code, in the INCREF/DECREF stuff, for example. The advantage with inline functions (where you can use them) is that the debugger can work with them. They are also more readable in my mind (no protective parens required around expressions/arguments, no do { ... } while (0) } business, no intrusive backslashification of every line) and they probably play nicer with editors (think Emacs speedbar or tags file - not sure if etags groks macros). In any case, I was just somewhat surprised to see relatively new code using macros where it seemed inline functions would have worked as well or better. My more general question stands. Should PEP 7 say something about the two? (Someone mentioned constants. Should they be preferred over macros?) Skip
04.12.19 22:12, Skip Montanaro пише:
This is my last post on this, at least as far as specific usage instances are concerned. See my question about PEP 7 below. If that is a discussion people think worthwhile, please start a new thread.
if (!VISIT(...)) { return 0; } if (!VISIT(...)) { return 0; } if (!VISIT(...)) { return 0; }
instead of just
VISIT(...); VISIT(...); VISIT(...);
That seems easily solved with the VISIT-as-macro calling _VISIT-as-inline function.
I do not understant what problem do you want to solve. VISIT() in symtable.c is: #define VISIT(ST, TYPE, V) \ if (!symtable_visit_ ## TYPE((ST), (V))) \ VISIT_QUIT((ST), 0); It literally calls other (non-inlined) function, check its result and returns from the caller function (in VISIT_QUIT). I do not understand how inline function can help here.
In any case, I was just somewhat surprised to see relatively new code using macros where it seemed inline functions would have worked as well or better.
In these cases macros cannot be replaced by inline functions because inline function do not have access to variables of the caller and cannot affect the control flow of the caller. The new code follows idioms used in the old code. There are also many other cases where macros save as from duplicating code and cannot be replaced with inline functions (C++ exceptions, templates, constructors and constant expressions could replace macros in some cases, but CPython is implemented on pure C, not even using all features of the recent standard). Don't afraid macros, they are the part of the language and pretty safe if use them properly.
On 04/12/2019 18:22, Serhiy Storchaka wrote:
In these files (symtable.c, compile.c, ast_opt.c, etc) there are sequences of calls for child nodes. Every call can return an error, so you need to check every call and return an error immediately after the call. With inline functions you would need to write
if (!VISIT(...)) { return 0; } if (!VISIT(...)) { return 0; } if (!VISIT(...)) { return 0; }
instead of just
VISIT(...); VISIT(...); VISIT(...);
Can I just say as a C programmer by trade that I hate this style of macro with the burning passion of a thousand fiery suns? Code like the second example is harder to comprehend because you can't simply see that it can change your flow of control. It comes as a surprise that if something went wrong in the first VISIT(), the remaining VISIT()s don't get called. It's not so bad in the case you've demonstrated of bombing out on errors, but I've seen the idiom used much less coherently in real-world applications to produce code that is effectively unreadable. It took me a long time to wrap my brain around what the low-level parsing code in Expat was doing, for example. I strongly recommend not starting down that path. -- Rhodri James *-* Kynesim Ltd
[Skip Montanaro <skip.montanaro@gmail.com>]
... I don't think stable code which uses macros should be changed (though I see the INCREF/DECREF macros just call private inline functions, so some conversion has clearly been done). Still, in new code, shouldn't the use of macros for more than trivial use cases (constant defs, simple one-liners) be discouraged at this point?
Within reason, I think so. Like C's old "register" declaration, compilers will eventually evolve to make better decisions about what should be done than humans generally do. But there are macros that exist just to reduce repetitive, error-prone typing, and others that set up control flow (like the trashcan macros. or listobject.c's IFLT). Which is the "within reason" part above. There are still fine uses for macros in C. It's just that faking inline functions is no longer one of them ;-)
Hi, I'm working on cleaning up the C API, long rationale: https://pythoncapi.readthedocs.io/ Macros are causing multiple issues: * They often leak "implementation details" and so are incompatible with a stable ABI * They have multiple pitfalls: https://gcc.gnu.org/onlinedocs/cpp/Macro-Pitfalls.html * They require hacks like "do { ... } while (0)" to behave properly as a statement, otherwise they can introduce bugs * They don't "check" argument types and their return type is often unclear, or worse depends on the arguments type * Variable scoping can be an issue. For example, Py_SETREF() macro in Python 3.8 uses a "_py_tmp" variable name, rather than being able to use a more common name like "op" or "obj". * etc. Static inline functions behave as regular functions: a function call is an expression, parameter types and return type are well defined, scoping is well defined by the C language, etc. For backward compatibility, I kept implicit cast to PyObject* using a macro. Example: static inline void _Py_INCREF(PyObject *op) { _Py_INC_REFTOTAL; op->ob_refcnt++; } #define Py_INCREF(op) _Py_INCREF(_PyObject_CAST(op)) Static inline functions like _Py_INCREF() are still incompatible with a stable ABI, but it's a tradeoff between correctness and practicability. I wrote an article about my work on the C API in Python 3.8: https://vstinner.github.io/split-include-directory-python38.html -- Macros are still used for some corner cases. For example, the following macro opens a block with { : #define Py_BEGIN_ALLOW_THREADS { \ PyThreadState *_save; \ _save = PyEval_SaveThread(); and this one closes the block: #define Py_END_ALLOW_THREADS PyEval_RestoreThread(_save); \ } Note also the "_save" variable which has a local scope ;-) Another example of special macro, the following macro uses "return" which cannot be used like that using a function: #define Py_RETURN_NONE return Py_INCREF(Py_None), Py_None By the way, I would prefer "return Py_GetNone()", but that's a different discussion ;-) -- Macros are sometimes overriden by #define, or even #undef followed by #define. In my own projects, I prefer "const int my_constant = 123;" rather than "#define constant 123". In Python, I'm using the status quo: "#define constant 123". Victor Le mer. 4 déc. 2019 à 13:19, Skip Montanaro <skip.montanaro@gmail.com> a écrit :
As I wander around the code base, I keep seeing macro definitions in the C code. For example, there are four CALL* macros defined in Python/ast_opt.c which contain not entirely trivial bits of syntax. That code is from 2017 (as compared to, say, Modules/audioop.c, which first saw the light of day in 1992) I see the inline keyword used unconditionally in many places.
I don't think stable code which uses macros should be changed (though I see the INCREF/DECREF macros just call private inline functions, so some conversion has clearly been done). Still, in new code, shouldn't the use of macros for more than trivial use cases (constant defs, simple one-liners) be discouraged at this point?
Skip _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AVF6W3PM... Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.
participants (5)
-
Rhodri James
-
Serhiy Storchaka
-
Skip Montanaro
-
Tim Peters
-
Victor Stinner