Improving xmlrpc introspection
Hello. This idea proposes enhancing the xmlrpc library by adding a couple of introspectable servers and proxies. For instance, here's an output of using the current idioms.
proxy = ServerProxy('http://localhost:8000') dir(proxy) ['_ServerProxy__allow_none', '_ServerProxy__close', '_ServerProxy__encoding', '_ServerProxy__handler', '_ServerProxy__host', '_ServerProxy__request', '_ServerProxy__transport', '_ServerProxy__verbose', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattr__' , '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']
Nothing useful in dir. The following works only if the server enables introspection:
proxy.system.listMethods() ['mul', 'pow', 'system.listMethods', 'system.methodHelp', 'system.methodSignature']
Now, let's see what mul does:
proxy.mul
help(proxy.mul) Help on _Method in module xmlrpc.client object:
class _Method(builtins.object) | Methods defined here: | | __call__(self, *args) | | __getattr__(self, name) | | __init__(self, send, name) | # some magic to bind an XML-RPC method to an RPC server. | # supports "nested" methods (e.g. examples.getStateName) | | ---------------------------------------------------------------------- | Data descriptors defined here: | | __dict__ | dictionary for instance variables (if defined) | | __weakref__ | list of weak references to the object (if defined) Nothing useful for us. Neither methodHelp, nor methodSignature are very useful:
proxy.system.methodHelp('mul') 'multiplication' proxy.system.methodSignature('mul') 'signatures not supported'
proxy.mul(1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1091, in __call__ return self.__send(self.__name, args) File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1421, in __request verbose=self.__verbose File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1133, in request return self.single_request(host, handler, request_body, verbose) File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1149, in single_request return self.parse_response(resp) File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1320, in parse_response return u.close() File "D:\Projects\cpython\lib\xmlrpc\client.py", line 658, in close raise Fault(**self._stack[0]) xmlrpc.client.Fault:
:mul() takes 3
We can find out something about that method by calling it. positional arguments but 4 were given"> So, only after calling a method, one can find meaningful informations about it. My idea behaves like this:
from xmlrpc.client import MagicProxy # not a very good name, but it does some magic behind proxy = MagicProxy('http://localhost:8000') dir(proxy) ['_ServerProxy__allow_none', '_ServerProxy__close', '_ServerProxy__encoding', '_ServerProxy__handler', '_ServerProxy__host', '_ServerProxy__request', '_ServerProxy__trans ', '_ServerProxy__verbose', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_collect_methods', '_original_mul', '_original_pow', 'mul', 'pow'] proxy.mul
proxy.pow help(proxy.mul) Help on function mul in module xmlrpc.client:
mul(x:1, y) -> 2 multiplication
help(proxy.pow) Help on function pow in module xmlrpc.client:
pow(*args, **kwargs) pow(x, y[, z]) -> number With two arguments, equivalent to x**y. With three arguments, equivalent to (x**y) % z, but may be more efficient (e.g. for ints).
proxy.mul(1) Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: mul() missing 1 required positional argument: 'y' proxy.mul(1, 2, 3) Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: mul() takes 2 positional arguments but 3 were given proxy.mul(1, 2) 2 import inspect inspect.signature(proxy.mul)
2">
As we can see, the registered methods can be introspected and calling one with the wrong number of arguments will not trigger a request to the server, but will fail right in the user's code. As a problem, it will work only for servers written in Python. For others will fallback to the current idiom. Would something like this be useful as an addition to the stdlib's xmlrpc module? If someone wants to test it, here's a rough patch against tip: https://gist.github.com/PCManticore/cf82ab421d4dc5c7f6ff. Thanks!
Since this is an internet client+server, can you please also consider
security as part of your design? Perhaps it's not always a good idea to
have that much introspectability on a web interface.
On Wed, Jun 18, 2014 at 3:23 AM, Claudiu Popa
Hello.
This idea proposes enhancing the xmlrpc library by adding a couple of introspectable servers and proxies. For instance, here's an output of using the current idioms.
proxy = ServerProxy('http://localhost:8000') dir(proxy) ['_ServerProxy__allow_none', '_ServerProxy__close', '_ServerProxy__encoding', '_ServerProxy__handler', '_ServerProxy__host', '_ServerProxy__request', '_ServerProxy__transport', '_ServerProxy__verbose', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattr__' , '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']
Nothing useful in dir. The following works only if the server enables introspection:
proxy.system.listMethods() ['mul', 'pow', 'system.listMethods', 'system.methodHelp', 'system.methodSignature']
Now, let's see what mul does:
proxy.mul
help(proxy.mul) Help on _Method in module xmlrpc.client object: class _Method(builtins.object) | Methods defined here: | | __call__(self, *args) | | __getattr__(self, name) | | __init__(self, send, name) | # some magic to bind an XML-RPC method to an RPC server. | # supports "nested" methods (e.g. examples.getStateName) | | ---------------------------------------------------------------------- | Data descriptors defined here: | | __dict__ | dictionary for instance variables (if defined) | | __weakref__ | list of weak references to the object (if defined)
Nothing useful for us. Neither methodHelp, nor methodSignature are very useful:
proxy.system.methodHelp('mul') 'multiplication' proxy.system.methodSignature('mul') 'signatures not supported'
We can find out something about that method by calling it.
proxy.mul(1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1091, in __call__ return self.__send(self.__name, args) File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1421, in __request verbose=self.__verbose File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1133, in request return self.single_request(host, handler, request_body, verbose) File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1149, in single_request return self.parse_response(resp) File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1320, in parse_response return u.close() File "D:\Projects\cpython\lib\xmlrpc\client.py", line 658, in close raise Fault(**self._stack[0]) xmlrpc.client.Fault:
:mul() takes 3 positional arguments but 4 were given"> So, only after calling a method, one can find meaningful informations about it. My idea behaves like this:
from xmlrpc.client import MagicProxy # not a very good name, but it does some magic behind proxy = MagicProxy('http://localhost:8000') dir(proxy) ['_ServerProxy__allow_none', '_ServerProxy__close', '_ServerProxy__encoding', '_ServerProxy__handler', '_ServerProxy__host', '_ServerProxy__request', '_ServerProxy__trans ', '_ServerProxy__verbose', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_collect_methods', '_original_mul', '_original_pow', 'mul', 'pow'] proxy.mul
proxy.pow help(proxy.mul) Help on function mul in module xmlrpc.client: mul(x:1, y) -> 2 multiplication
help(proxy.pow) Help on function pow in module xmlrpc.client:
pow(*args, **kwargs) pow(x, y[, z]) -> number
With two arguments, equivalent to x**y. With three arguments, equivalent to (x**y) % z, but may be more efficient (e.g. for ints).
proxy.mul(1) Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: mul() missing 1 required positional argument: 'y' proxy.mul(1, 2, 3) Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: mul() takes 2 positional arguments but 3 were given proxy.mul(1, 2) 2 import inspect inspect.signature(proxy.mul)
2"> As we can see, the registered methods can be introspected and calling one with the wrong number of arguments will not trigger a request to the server, but will fail right in the user's code. As a problem, it will work only for servers written in Python. For others will fallback to the current idiom. Would something like this be useful as an addition to the stdlib's xmlrpc module? If someone wants to test it, here's a rough patch against tip: https://gist.github.com/PCManticore/cf82ab421d4dc5c7f6ff.
Thanks! _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
On 19 Jun 2014 00:44, "Guido van Rossum"
Since this is an internet client+server, can you please also consider
security as part of your design? Perhaps it's not always a good idea to have that much introspectability on a web interface. I don't recall the details, but there was a CVE quite some time ago for an information leak in SimpleXMLRPCServer. That's not necessarily a "this is a bad idea" response, just "the security implications of such a feature would need to be managed very carefully (and if that's too hard to do, it might be a bad idea)". Cheers, Nick.
On Wed, Jun 18, 2014 at 3:23 AM, Claudiu Popa
wrote:
Hello.
This idea proposes enhancing the xmlrpc library by adding a couple of introspectable servers and proxies. For instance, here's an output of using the current idioms.
proxy = ServerProxy('http://localhost:8000') dir(proxy) ['_ServerProxy__allow_none', '_ServerProxy__close', '_ServerProxy__encoding', '_ServerProxy__handler', '_ServerProxy__host', '_ServerProxy__request', '_ServerProxy__transport', '_ServerProxy__verbose', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattr__' , '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']
Nothing useful in dir. The following works only if the server enables introspection:
proxy.system.listMethods() ['mul', 'pow', 'system.listMethods', 'system.methodHelp', 'system.methodSignature']
Now, let's see what mul does:
proxy.mul
help(proxy.mul) Help on _Method in module xmlrpc.client object: class _Method(builtins.object) | Methods defined here: | | __call__(self, *args) | | __getattr__(self, name) | | __init__(self, send, name) | # some magic to bind an XML-RPC method to an RPC server. | # supports "nested" methods (e.g. examples.getStateName) | |
| Data descriptors defined here: | | __dict__ | dictionary for instance variables (if defined) | | __weakref__ | list of weak references to the object (if defined)
Nothing useful for us. Neither methodHelp, nor methodSignature are very useful:
proxy.system.methodHelp('mul') 'multiplication' proxy.system.methodSignature('mul') 'signatures not supported'
We can find out something about that method by calling it.
proxy.mul(1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1091, in __call__ return self.__send(self.__name, args) File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1421, in __request verbose=self.__verbose File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1133, in request return self.single_request(host, handler, request_body, verbose) File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1149, in single_request return self.parse_response(resp) File "D:\Projects\cpython\lib\xmlrpc\client.py", line 1320, in parse_response return u.close() File "D:\Projects\cpython\lib\xmlrpc\client.py", line 658, in close raise Fault(**self._stack[0]) xmlrpc.client.Fault:
:mul() takes 3 positional arguments but 4 were given"> So, only after calling a method, one can find meaningful informations about it. My idea behaves like this:
from xmlrpc.client import MagicProxy # not a very good name, but it does some magic behind proxy = MagicProxy('http://localhost:8000') dir(proxy) ['_ServerProxy__allow_none', '_ServerProxy__close', '_ServerProxy__encoding', '_ServerProxy__handler', '_ServerProxy__host', '_ServerProxy__request', '_ServerProxy__trans ', '_ServerProxy__verbose', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_collect_methods', '_original_mul', '_original_pow', 'mul', 'pow'] proxy.mul
proxy.pow help(proxy.mul) Help on function mul in module xmlrpc.client: mul(x:1, y) -> 2 multiplication
help(proxy.pow) Help on function pow in module xmlrpc.client:
pow(*args, **kwargs) pow(x, y[, z]) -> number
With two arguments, equivalent to x**y. With three arguments, equivalent to (x**y) % z, but may be more efficient (e.g. for ints).
proxy.mul(1) Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: mul() missing 1 required positional argument: 'y' proxy.mul(1, 2, 3) Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: mul() takes 2 positional arguments but 3 were given proxy.mul(1, 2) 2 import inspect inspect.signature(proxy.mul)
2"> As we can see, the registered methods can be introspected and calling one with the wrong number of arguments will not trigger a request to the server, but will fail right in the user's code. As a problem, it will work only for servers written in Python. For others will fallback to the current idiom. Would something like this be useful as an addition to the stdlib's xmlrpc module? If someone wants to test it, here's a rough patch against tip: https://gist.github.com/PCManticore/cf82ab421d4dc5c7f6ff.
Thanks! _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
I might be a bit confused (nothing new there), but it seemed to me that Claudiu indicated all his MagicProxy magic happens in the client:
As we can see, the registered methods can be introspected and calling one with the wrong number of arguments will not trigger a request to the server, but will fail right in the user's code.
I think we will have to see the code to decide if it's a security risk. Claudiu, I suggest you open an issue in the tracker so others can see how the magic works. Skip
On Wed, Jun 18, 2014 at 6:52 PM, Skip Montanaro
I might be a bit confused (nothing new there), but it seemed to me that Claudiu indicated all his MagicProxy magic happens in the client:
As we can see, the registered methods can be introspected and calling one with the wrong number of arguments will not trigger a request to the server, but will fail right in the user's code.
I think we will have to see the code to decide if it's a security risk. Claudiu, I suggest you open an issue in the tracker so others can see how the magic works.
Skip _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
That's right, the behaviour occurs in the client, the only catch is that it needs a new method in xmlrpc.server and the server must support introspection already, by providing the `system` proxy methods. I already posted a sample patch in the first message (https://gist.github.com/PCManticore/cf82ab421d4dc5c7f6ff). Now, something is wrong in the client, because it exec's the information received in order to create the local functions, but probably there are other methods for achieving the same behaviour. Anyway, thank you all for your responses. I admit that I didn't think at the security implications of this proposal very much and it was enlightening as is.
participants (4)
-
Claudiu Popa
-
Guido van Rossum
-
Nick Coghlan
-
Skip Montanaro