Should the use of PyArg_NoArgs() be deprecated? There are many uses (53) throughout Modules/*.c. It seems that this check is not useful anymore if the MethodDef is set to METH_NOARGS. Is this correct? If so, I can make a patch. Neal
Neal Norwitz writes:
Should the use of PyArg_NoArgs() be deprecated?
Probably. Even if used with METH_OLDARSG, it doesn't produce as nice an error message because it doesn't know the name of the function it is being called for. There should be strong encouragement to shift to using METH_NOARGS in code that doesn't need to be compatible with older versions of Python. (And the code distributed with Python certainly seems to fall into that category!)
There are many uses (53) throughout Modules/*.c. It seems that this check is not useful anymore if the MethodDef is set to METH_NOARGS. Is this correct? If so, I can make a patch.
Please do. Thanks! -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation
Should the use of PyArg_NoArgs() be deprecated?
Sure, why not.
There are many uses (53) throughout Modules/*.c. It seems that this check is not useful anymore if the MethodDef is set to METH_NOARGS.
Yes, that's the recommended way.
Is this correct? If so, I can make a patch.
You can just check it in. --Guido van Rossum (home page: http://www.python.org/~guido/)
Neal Norwitz <neal@metaslash.com> writes:
Should the use of PyArg_NoArgs() be deprecated?
There are many uses (53) throughout Modules/*.c. It seems that this check is not useful anymore if the MethodDef is set to METH_NOARGS. Is this correct? If so, I can make a patch.
I would prefer if you could simultaneously remove occurences of METH_OLDARGS (introducing METH_O where possible), so that modules get cleaned-up entirely, rather than just replacing the easy parts :-) The reason why such changes haven't been applied to all modules is that some of the modules are old in themselves, and I would have no way to verify that the module is working properly after the changes. Regards, Martin
"Martin v. Loewis" wrote:
Neal Norwitz <neal@metaslash.com> writes:
Should the use of PyArg_NoArgs() be deprecated?
There are many uses (53) throughout Modules/*.c. It seems that this check is not useful anymore if the MethodDef is set to METH_NOARGS. Is this correct? If so, I can make a patch.
I would prefer if you could simultaneously remove occurences of METH_OLDARGS (introducing METH_O where possible), so that modules get cleaned-up entirely, rather than just replacing the easy parts :-)
Ok, I guess I can go a little further... :-) There are about 238 uses of METH_OLDARGS, all but 2 are in Modules/. I don't know how to fix the other 2: Objects/fileobject.c: {"readinto", METH_OLDARGS, Python/sysmodule.c: {"exit", METH_OLDARGS, But I will take a look at the other Modules.
The reason why such changes haven't been applied to all modules is that some of the modules are old in themselves, and I would have no way to verify that the module is working properly after the changes.
Yes, this could be a problem, but it is very early on in 2.3. But if the modules aren't used at all, no one will notice. :-) I will start by checking in all the more current modules: _curses_panel, _localemodule, bsddbmodule, md5module, pwdmodule, readline, signalmodule, threadmodule Other modules which I've fixed, but can't test: clmodule, flmodule, fmmodule, glmodule, nismodule nis should be testable by someone. On a related note, PyArg_GetInt() macro is used exactly one time-- in socketmodule.c There seem to be many more places where this could be used. Should I use or remove the macro? Or doc that it should not be used? Neal
I don't know how to fix the other 2:
Objects/fileobject.c: {"readinto", METH_OLDARGS,
This seems a simple case of replacing PyArg_Parse with PyArg_ParseTuple.
Python/sysmodule.c: {"exit", METH_OLDARGS,
This one *appears* tricky. But in fact, changing it to METH_VARARGS should work just as well (I think).
nis should be testable by someone.
Or deprecated. I've never met somebody who used it.
On a related note, PyArg_GetInt() macro is used exactly one time-- in socketmodule.c There seem to be many more places where this could be used. Should I use or remove the macro? Or doc that it should not be used?
It's deprecated, just as PyArg_NoArgs(). Now's a good time to get rid of the last usage instance. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum <guido@python.org> writes:
Python/sysmodule.c: {"exit", METH_OLDARGS,
This one *appears* tricky. But in fact, changing it to METH_VARARGS should work just as well (I think).
I agree, but a comment that args will be always a tuple seems appropriate.
nis should be testable by someone.
Or deprecated. I've never met somebody who used it.
You met me :-) I found a need to parse auto.home on Linux, once. Regards, Martin
Guido van Rossum wrote:
Python/sysmodule.c: {"exit", METH_OLDARGS,
This one *appears* tricky. But in fact, changing it to METH_VARARGS should work just as well (I think).
The only problem I ran into just using METH_VARARGS is that the parameter is always printed. So I used PyArg_ParseTuple() and passed that object to PyErr_SetObject(PyExc_SystemExit, exit_code). This works as before wrt printing. However, you can no longer pass an arbitrary number of parameters, either 0 or 1. Is this ok?
On a related note, PyArg_GetInt() macro is used exactly one time-- in socketmodule.c There seem to be many more places where this could be used. Should I use or remove the macro? Or doc that it should not be used?
It's deprecated, just as PyArg_NoArgs(). Now's a good time to get rid of the last usage instance.
Both PyArg_GetInt() & PyArg_NoArgs() both have comments above them stating they should not be used. There are no more occurances of PyArg_GetInt(). I'm working on reducing METH_OLDARGS, down to 214. If you want I can remove them from Include/Python.h when done. If so, should I write up anything in NEWS? Neal
Guido van Rossum wrote:
Python/sysmodule.c: {"exit", METH_OLDARGS,
This one *appears* tricky. But in fact, changing it to METH_VARARGS should work just as well (I think).
[Neal]
The only problem I ran into just using METH_VARARGS is that the parameter is always printed.
Oops. That's wrong.
So I used PyArg_ParseTuple() and passed that object to PyErr_SetObject(PyExc_SystemExit, exit_code). This works as before wrt printing. However, you can no longer pass an arbitrary number of parameters, either 0 or 1. Is this ok?
It's documented as having one optional arg. It might break code if we disallow multiple args, but I don't think I care. So, yes, OK.
On a related note, PyArg_GetInt() macro is used exactly one time-- in socketmodule.c There seem to be many more places where this could be used. Should I use or remove the macro? Or doc that it should not be used?
It's deprecated, just as PyArg_NoArgs(). Now's a good time to get rid of the last usage instance.
Both PyArg_GetInt() & PyArg_NoArgs() both have comments above them stating they should not be used. There are no more occurances of PyArg_GetInt(). I'm working on reducing METH_OLDARGS, down to 214.
If you want I can remove them from Include/Python.h when done.
Has the api documentation documented them as obsolete? If so, or if they were undocumented, yes, please remove them. If not, I'd see if there's a way to turn these into warnings for one release.
If so, should I write up anything in NEWS?
Yes, I think so. It's always good to announce explicitly when a previously deprecated item actually gets deleted, no matter how small. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Both PyArg_GetInt() & PyArg_NoArgs() both have comments above them stating they should not be used. There are no more occurances of PyArg_GetInt(). I'm working on reducing METH_OLDARGS, down to 214.
If you want I can remove them from Include/Python.h when done.
Has the api documentation documented them as obsolete? If so, or if they were undocumented, yes, please remove them. If not, I'd see if there's a way to turn these into warnings for one release.
+1 on removing PyArg_GetInt() -1 on removing PyArg_NoArgs() ... this API is still in very active use in third party extensions.
If so, should I write up anything in NEWS?
Yes, I think so. It's always good to announce explicitly when a previously deprecated item actually gets deleted, no matter how small.
I think we should come up with a better way to deprecate C level APIs. NEWS is not as effective as one might wish; any deprecation or removal of an API should also include upgrade information for the C extension writer so that upgrading their modules does not require diving into the new style C API code every time. I'd suggest to have a PEP which documents all C API changes or deprecations together with a short decsription of how to update old code. Another aspect which we ought to consider is that C extension writers may have a different view of when to leave backward compatibility path in favour of a new Python version. IMHO, there should always be a way to emulate old behaviour so that extensions can continue to support Python 1.5.2 if they need to. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
"M.-A. Lemburg" wrote:
I think we should come up with a better way to deprecate C level APIs. NEWS is not as effective as one might wish; any deprecation or removal of an API should also include upgrade information for the C extension writer so that upgrading their modules does not require diving into the new style C API code every time.
I'd suggest to have a PEP which documents all C API changes or deprecations together with a short decsription of how to update old code.
I think a PEP will be ignored just as easily as NEWS. Perhaps, we could wrap all deprecated functions/macros in: #ifdef Py_DEPRECATED #endif So if an extension wants to use deprecated functions, they would need to #define Py_DEPRECATED. We could also use another macro Py_OBSOLETE. In this way functions/macros would move into Py_DEPRECATED, then Py_OBSOLETE, then removed. The functions would still be in a library. And when a function is used a warning would be produced since there is no prototype. Not sure how to handle macros, should they remain undefined and create errors? If we wanted to be more heavy handed, we could create errors that would force extensions to be compiled w/Py_DEPRECATED by doing: #ifdef Py_DEPRECATED #define Py_MACRO(a) do_something(a) int Py_prototype(...); #else /* Create a syntax error unless Py_DEPRECATED is defined */ #define Py_MACRO(a) === #define Py_prototype === #endif
Another aspect which we ought to consider is that C extension writers may have a different view of when to leave backward compatibility path in favour of a new Python version. IMHO, there should always be a way to emulate old behaviour so that extensions can continue to support Python 1.5.2 if they need to.
Perhaps, we could put the deprecated functions into a libpythoncompat.a. It would probably be best to move the functions out of the code they are in and into a new compat.c module. I'm not sure how workable this idea is. Neal
Neil Schemenauer wrote:
Neal Norwitz wrote:
Perhaps, we could wrap all deprecated functions/macros in:
#ifdef Py_DEPRECATED #endif
I like this idea. We could easily undefine it to ensure that the core code is not using deprecated APIs.
FYI, I have been using a mxpyapi.h header file for this the last couple of years (see e.g. mxDateTime). Perhaps Python should ship with one of these as well, e.g. pycompat.h. This include file would only be used by extensions wishing to support older Python versions, of course, not by the Python core itself. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
+1 on removing PyArg_GetInt()
OK.
-1 on removing PyArg_NoArgs() ... this API is still in very active use in third party extensions.
Would it be OK if it issued a warning?
I think we should come up with a better way to deprecate C level APIs. NEWS is not as effective as one might wish; any deprecation or removal of an API should also include upgrade information for the C extension writer so that upgrading their modules does not require diving into the new style C API code every time.
In this case, PyArg_ParseTuple(args, ":functionName") is pretty simple to explain.
I'd suggest to have a PEP which documents all C API changes or deprecations together with a short decsription of how to update old code.
It shouldn't be a PEP (that's a *proposal*). It should be part of the C API docs, possibly an appendix.
Another aspect which we ought to consider is that C extension writers may have a different view of when to leave backward compatibility path in favour of a new Python version. IMHO, there should always be a way to emulate old behaviour so that extensions can continue to support Python 1.5.2 if they need to.
Not a problem in this case. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
+1 on removing PyArg_GetInt()
OK.
-1 on removing PyArg_NoArgs() ... this API is still in very active use in third party extensions.
Would it be OK if it issued a warning?
Hmm, people would start screaming, I guess -- this API is used far more than e.g. the regex/regsub module combo. A one-time-per- module warning would probably be OK though. I seriously think that we should install a set of common third party extensions somewhere to that we can easily check usage of the APIs in the outside world, e.g. take the packages Aahz collected recently and unzip them to some directory on python.org. A grep would then quickly produce proof of "common" usage.
I think we should come up with a better way to deprecate C level APIs. NEWS is not as effective as one might wish; any deprecation or removal of an API should also include upgrade information for the C extension writer so that upgrading their modules does not require diving into the new style C API code every time.
In this case, PyArg_ParseTuple(args, ":functionName") is pretty simple to explain.
I'd suggest to have a PEP which documents all C API changes or deprecations together with a short decsription of how to update old code.
It shouldn't be a PEP (that's a *proposal*). It should be part of the C API docs, possibly an appendix.
I was thinking of an informational PEP -- mostly because PEPs are easily accessible and also simple to write (LaTeX is not everybody's friend ;-).
Another aspect which we ought to consider is that C extension writers may have a different view of when to leave backward compatibility path in favour of a new Python version. IMHO, there should always be a way to emulate old behaviour so that extensions can continue to support Python 1.5.2 if they need to.
Not a problem in this case.
True, but there are other instances where this was a problem in the past, e.g. #if PY_VERSION_HEX < 0x01060000 #if !defined(PyObject_DEL) # define PyObject_DEL(x) free(x) # define PyObject_Del(x) free(x) #endif #endif -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
On Tue, Apr 02, 2002, M.-A. Lemburg wrote:
I seriously think that we should install a set of common third party extensions somewhere to that we can easily check usage of the APIs in the outside world, e.g. take the packages Aahz collected recently and unzip them to some directory on python.org. A grep would then quickly produce proof of "common" usage.
In case anyone wants to take up MAL's idea, here's an alphabetical list of all the extensions I recognized in that thread: 4Suite calldll Gadfly HTMLgen mxODBC NumPy piddle PIL PyGame PyOpenGL PythonCard SWIG win32all wxPython ZODB -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups?
Aahz wrote:
On Tue, Apr 02, 2002, M.-A. Lemburg wrote:
I seriously think that we should install a set of common third party extensions somewhere to that we can easily check usage of the APIs in the outside world, e.g. take the packages Aahz collected recently and unzip them to some directory on python.org. A grep would then quickly produce proof of "common" usage.
In case anyone wants to take up MAL's idea, here's an alphabetical list of all the extensions I recognized in that thread:
4Suite calldll Gadfly HTMLgen mxODBC NumPy piddle PIL PyGame PyOpenGL PythonCard SWIG win32all wxPython ZODB
Thanks, I'll setup a source archive directory on python.org. Hmm, you don't happen have a similar list of URLs for the source archives... :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
-1 on removing PyArg_NoArgs() ... this API is still in very active use in third party extensions.
Would it be OK if it issued a warning?
Hmm, people would start screaming, I guess -- this API is used far more than e.g. the regex/regsub module combo. A one-time-per- module warning would probably be OK though.
That's what I wanted, but now that you mention it I'm not sure if the warning module can currrently detect from which module a particular C API is called -- it'll probably put the blame on a particular line in the user's code, and then repeat the warning for each *different* line of user code.
I seriously think that we should install a set of common third party extensions somewhere to that we can easily check usage of the APIs in the outside world, e.g. take the packages Aahz collected recently and unzip them to some directory on python.org. A grep would then quickly produce proof of "common" usage.
+1
I'd suggest to have a PEP which documents all C API changes or deprecations together with a short decsription of how to update old code.
It shouldn't be a PEP (that's a *proposal*). It should be part of the C API docs, possibly an appendix.
I was thinking of an informational PEP -- mostly because PEPs are easily accessible and also simple to write (LaTeX is not everybody's friend ;-).
Yeah, that's fine. Let whoever writes it decide on the format. :-)
True, but there are other instances where this was a problem in the past, e.g.
#if PY_VERSION_HEX < 0x01060000
#if !defined(PyObject_DEL) # define PyObject_DEL(x) free(x) # define PyObject_Del(x) free(x) #endif
#endif
Why the PY_VERSION_HEX guard? Isn't the !defined() guard enough? --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum <guido@python.org> writes:
That's what I wanted, but now that you mention it I'm not sure if the warning module can currrently detect from which module a particular C API is called -- it'll probably put the blame on a particular line in the user's code, and then repeat the warning for each *different* line of user code.
What *could* be done, of course, is to give the warning at most once. Regards, Martin
"Martin v. Loewis" wrote:
Guido van Rossum <guido@python.org> writes:
That's what I wanted, but now that you mention it I'm not sure if the warning module can currrently detect from which module a particular C API is called -- it'll probably put the blame on a particular line in the user's code, and then repeat the warning for each *different* line of user code.
What *could* be done, of course, is to give the warning at most once.
If that's possible, I'm +1 on generating a warning for the first usage of PyArg_NoArgs(). Would be nice to have the warning include the calling module name -- don't know whether that's possible, though (I have next to no experience with the warning framework). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
If that's possible, I'm +1 on generating a warning for the first usage of PyArg_NoArgs(). Would be nice to have the warning include the calling module name -- don't know whether that's possible, though (I have next to no experience with the warning framework).
It peeks in the current globals for __name__. The current globals are those of the most recent Python module on the stack. Since it also gives a line number, you might be able to track down which call it's referring to. But it might be somewhat surprising. --Guido van Rossum (home page: http://www.python.org/~guido/)
Martin v. Loewis wrote:
That's what I wanted, but now that you mention it I'm not sure if the warning module can currrently detect from which module a particular C API is called -- it'll probably put the blame on a particular line in the user's code, and then repeat the warning for each *different* line of user code.
What *could* be done, of course, is to give the warning at most once.
and what good is that, really? why would the *users* of a module I've written have to care about this? the Py_DEPRECATED approach (and the "grep" tool) is a much better way to use everybody's time. </F>
"Fredrik Lundh" <fredrik@pythonware.com> writes:
and what good is that, really? why would the *users* of a module I've written have to care about this?
the Py_DEPRECATED approach (and the "grep" tool) is a much better way to use everybody's time.
Your users will care about it because your module fails to compile if the deprecated API is wrapped with Py_DEPRECATED, whereas it continues to compile and run when the warning is produced - the warning is annoying (and really meant for you only), but is the lesser evil. Regards, Martin
Martin v. Loewis wrote:
"Fredrik Lundh" <fredrik@pythonware.com> writes:
and what good is that, really? why would the *users* of a module I've written have to care about this?
the Py_DEPRECATED approach (and the "grep" tool) is a much better way to use everybody's time.
Your users will care about it because your module fails to compile if the deprecated API is wrapped with Py_DEPRECATED
oh, please. 70-80% of my users use libraries that someone else compiled for them. giving meaningless warnings is a great way to make them question the quality of the software (I had enough of that when 2.1 started complaining about shadowed globals), but it won't help them a bit.
whereas it continues to compile and run when the warning is produced - the warning is annoying (and really meant for you only), but is the lesser evil.
if the message is meant for me, please make sure I see it. the C compiler/linker has no problem doing that. but I cannot guarantee that every library I've ever written has a test suite that tests every piece of the library, or that the test suite is set up to look for warnings (doctest won't catch them, for example). </F>
Guido van Rossum wrote:
I'd suggest to have a PEP which documents all C API changes or deprecations together with a short decsription of how to update old code.
It shouldn't be a PEP (that's a *proposal*). It should be part of the C API docs, possibly an appendix.
I was thinking of an informational PEP -- mostly because PEPs are easily accessible and also simple to write (LaTeX is not everybody's friend ;-).
Yeah, that's fine. Let whoever writes it decide on the format. :-)
Good idea .. as long as someone does :-)
True, but there are other instances where this was a problem in the past, e.g.
#if PY_VERSION_HEX < 0x01060000
#if !defined(PyObject_DEL) # define PyObject_DEL(x) free(x) # define PyObject_Del(x) free(x) #endif
#endif
Why the PY_VERSION_HEX guard? Isn't the !defined() guard enough?
Well, yes... this is more to document when the change happened. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
"Martin v. Loewis" wrote:
I would prefer if you could simultaneously remove occurences of METH_OLDARGS (introducing METH_O where possible), so that modules get cleaned-up entirely, rather than just replacing the easy parts :-)
I've finished the first step of getting rid of many uses of METH_OLDARGS. The bad news is that there are many more uses of METH_OLDARGS that were implicit in the declaration. Here's the current count of uses of METH_OLDARGS: Modules/audioop.c 22 Modules/clmodule.c 42 Modules/flmodule.c 103 Modules/fmmodule.c 6 Modules/glmodule.c 430 Modules/imageop.c 15 Modules/mpzmodule.c 8 Modules/sgimodule.c 2 Modules/_sre.c 2 Modules/stropmodule.c 7 Modules/svmodule.c 34 Modules/_tkinter.c 3 I will try to go through and get rid of more uses. I don't believe there are any uses outside of Modules/ now. Sorry if I broke anything. Neal
Neal Norwitz <neal@metaslash.com> writes:
I've finished the first step of getting rid of many uses of METH_OLDARGS.
Good work! I could not spot errors in you changes. For mpz, it might be useful to get rid of MPZ_CONVERSIONS_AS_METHODS altogether (probably of a few others as well, but that would be more difficult to analyse). Regards, Martin
participants (8)
-
Aahz -
Fred L. Drake, Jr. -
Fredrik Lundh -
Guido van Rossum -
M.-A. Lemburg -
martin＠v.loewis.de -
Neal Norwitz -
Neil Schemenauer