Proposal: Python Native bindings for OpenAPI
OpenAPI (https://spec.openapis.org/oas/latest.html) is an emerging standard for defining RESTful interfaces. OpenAPI is supported by swagger - which provides tools to convert an OpenAPI spec into python code (for both backend as well as client). The proposition I have is to make OpenAPI native to python. Essentially, this involves some of the following: 1. OpenAPI provides definitions of objects, arrays, structures, primitive types, collections etc. 2. OpenAPI supports object creation (HTTPS POST), object modification (PUT), object deletion (DELETE) and object retrieval (GET). 3. OpenAPI also supports operations for collections (add to collection, remove from collection and modify collection) 4. OpenAPI also supports methods and allows parameters to be passed to these APIs. The Native Python implementation would load in OpenAPI specification and provide native Python experience as follows: 1. Primitive types are mapped to python primitive types 2. Objects and structures are treated as classes or dicts wrapped. The field names in the Spec are accessible. 3. Arrays and Collections are mapped to python lists. 4. URLs are converted into object structure. 5. Python inspection methods (__getattr__, __setattr__, __new__, __del__ and others) of this model is changed to manipulate the fetched objects dynamically Parts of this implementations are found in various places : sushy (OpenStack) etc. However, having a more organized approach would make life easier. Swagger has a tool that can generate the code. But this means that every time a specification changes you need to ship a new python library. The methodology presented not require any addition python libraries - the tool can download from the REST end point. By making OpenAPI native to python, we would avoid cumbersome handling of request codes etc. and make it as you we are accessing native objects. I can provide more details and a reference implementation I am working on. Wanted to know if this is worthy to be submitted as PEP!
You may want to take a look at FastAPI[1], it already does more or less what you ask for. There is no need for it to be part of the Python core, it's fine as it is as a package. [1] https://fastapi.tiangolo.com/ On Fri, 6 Aug 2021 at 17:39, Vaideeswaran Ganesan <vaidees@gmail.com> wrote:
-- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert
I looked at the fastapi code. I actually want to avoid get, post, put, 2xx, 4xx codes in the client portions of the code. (I don't think it's pythonic). And Fastapi seems to have them exactly the same. Look at how the code looks when you are using FastAPI and with my OpenAPI Native Bindings - below: =============== [Code using FastAPI] def test_create_item(): response = client.post( "/items/", headers={ "X-Token": "coneofsilence"}, json={"id": "foobar", "title": "Foo Bar", "description": "The Foo Barters"}, ) assert response.status_code == 200 assert response.json() == { "id": "foobar", "title": "Foo Bar", "description": "The Foo Barters", }. =============== [Code using pyopenapi - my proposal] def test_create_item(): client = client('host', creds).connect('/') try: assert client.items.id == "foobar" assert client.items.title == "Foo Bar" assert client.items.description == "The Foo Barters" except AttributeError: print("Items does not exist!") On Fri, Aug 6, 2021 at 10:13 PM Gustavo Carneiro <gjcarneiro@gmail.com> wrote:
In your code example it looks like FastAPI is making 1 HTTP request vs. your library is making 3 HTTP requests? Or are there some missing lines? Or am I missing something? Also are you referring to https://github.com/pyopenapi/pyopenapi ? And if so what would be the reasoning to pull something like this into the standard library rather than leaving it as a third part library? Damian (he / him) On Sat, Aug 7, 2021 at 12:40 AM Vaideeswaran Ganesan <vaidees@gmail.com> wrote:
Vaideeswaran Ganesan writes:
I actually want to avoid get, post, put, 2xx, 4xx codes in the client portions of the code.
I think this goal is too high-level for the standard library. I don't know what you expect on the other side, but in an application I work on it matters whether you're using GET or POST, whether you use POST, PUT, or PATCH in the application's REST API. I don't see using something other than those identifiers in the API, because they have well-defined semantics. If OpenAPI uses different identifiers, I would very likely avoid using it. The (apparent) mapping of AttributeError to "not 200" is pretty questionable, too. I could see providing an IntEnum for the standard status codes, but avoiding them entirely and instead catching a Python Error that doesn't seem to have an obvious mapping to the API's statuses doesn't seem like a great idea. If you conceal the status codes, what is the client supposed to do with an application-specific code, or a new code added to the standard?
(I don't think it's pythonic).
Not clear what you mean by that. Saying what you mean in the established language for those semantics seems Pythonic enough to me.
Not gonna fix the garbage text/plain from your mail client, but even so
looks more useful than
to me. Also, the two examples seem to have very different semantics. The first not only POSTs the items but also presents some sort of auxiliary data (the X-Token). The second appears to be implicitly a fetch despite the function name, but whether it's a GET or a POST with implicitly specified content, and what's in "creds" is quite unclear; I guess that's probably the X-Token again, but whether that is going to be the X-Token HTTP header or part of the POST content is anybody's guess. If you want to make a convincing argument you're going to have to provide better examples and more discussion of them.
It is converting the REST API syntax into language constructs. If such mapping doesn't exist - I agree with you that we probably should give flexibility to user. On the other hand, if such mapping exists and I strongly believe so - seeing several examples in web, we should allow it. Intent is to make a web interaction as if you're working with local function. This is very well known design pattern called Proxy. It also simplifies the programmer allowing same experience whether you are working with local or remote interfaces. I know we are used to one paradigm. But sometimes, we need to see if others also are applicable in relevant cases. Given the examples I have seen (swagger, sushi), we should see where this could be relevant. On Sat, Aug 7, 2021, 13:14 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Vaideeswaran Ganesan writes:
Intent is to make a web interaction as if you're working with local function. This is very well known design pattern called Proxy.
Sure, but the stdlib is fairly low-level. The question is not whether this module is useful. I'm sure for some purposes it is very useful. The question is is it useful for enough common Python tasks, or is it so likely to be frequently implemented incorrectly, that we should provide *and maintain indefinitely* this module. This is a big problem for this particular module. "Maintenance" often is more than bug-fixing, it includes developing new features. I'm pretty sure that this module would attract a very large number of requests for enhancement, often contradictory (eg, "implicit fetches should use GET, vs. implicit fetches should use POST"). I don't think that's a good use of core resources as the proposal has been presented so far.
It also simplifies the programmer allowing same experience whether you are working with local or remote interfaces.
Yes. Please don't assume people who are unfavorable to your proposal are unaware of its theoretical advantages. Show us parallel code examples with at least very close semantics. If you won't/can't do that much, many of us will make very unfavorable assumptions about your willingness/ability to maintain the module (and somebody needs to promise to do that, with you the obvious candidate).
Given the examples I have seen (swagger, sushi), we should see where this could be relevant.
That logic is invalid for Python. What we *should* do is make sure that Python works as documented (the fundamental contract of any software) and provides the *necessary* tools to get work done (the Python-specific "batteries included" policy). We provide the hammer and saw, which are not so easy to make in your home workshop. You use those tools to build a workbench. As useful as a workbench is, Python doesn't need to provide it once it does provide the hammer and saw. Obviously, this is a matter of judgment. Is pyopenapi more like a hammer or a workbench? At first glance, I say it's like a workbench. It's your job to convince this list (you don't need to convince me personally), and eventually enough core committers, that it's enough like a hammer to be included. As for swagger and sushi, I haven't seen them, and I don't intend to look at them because I have more than enough frameworks that I'm already using that I'm dangerously underinformed about. Where I have time, I study them, I don't look for more trouble elsewhere. So presenting information about examples is *your job as proponent*. The *default* is to continue the status quo (you can freely contribute the module on PyPI, which is not at all a bad deal). *You* need to show that this module benefits *Python* if included; Python needs no reason at all to say "put it on PyPI, please, so if I ever want it I can find it easily". Steve
Thanks Steve, This is very useful and actionable feedback. I'll work on the parallel examples. I'm pretty much sure, it might take few rounds of consultation to bring out the value. I will also seriously think through the PyPI as an approach, as I seem to feel that OpenAPI is not really "all that universal". Regards, Vaidees. On Sun, Aug 8, 2021 at 8:34 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
You may want to take a look at FastAPI[1], it already does more or less what you ask for. There is no need for it to be part of the Python core, it's fine as it is as a package. [1] https://fastapi.tiangolo.com/ On Fri, 6 Aug 2021 at 17:39, Vaideeswaran Ganesan <vaidees@gmail.com> wrote:
-- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert
I looked at the fastapi code. I actually want to avoid get, post, put, 2xx, 4xx codes in the client portions of the code. (I don't think it's pythonic). And Fastapi seems to have them exactly the same. Look at how the code looks when you are using FastAPI and with my OpenAPI Native Bindings - below: =============== [Code using FastAPI] def test_create_item(): response = client.post( "/items/", headers={ "X-Token": "coneofsilence"}, json={"id": "foobar", "title": "Foo Bar", "description": "The Foo Barters"}, ) assert response.status_code == 200 assert response.json() == { "id": "foobar", "title": "Foo Bar", "description": "The Foo Barters", }. =============== [Code using pyopenapi - my proposal] def test_create_item(): client = client('host', creds).connect('/') try: assert client.items.id == "foobar" assert client.items.title == "Foo Bar" assert client.items.description == "The Foo Barters" except AttributeError: print("Items does not exist!") On Fri, Aug 6, 2021 at 10:13 PM Gustavo Carneiro <gjcarneiro@gmail.com> wrote:
In your code example it looks like FastAPI is making 1 HTTP request vs. your library is making 3 HTTP requests? Or are there some missing lines? Or am I missing something? Also are you referring to https://github.com/pyopenapi/pyopenapi ? And if so what would be the reasoning to pull something like this into the standard library rather than leaving it as a third part library? Damian (he / him) On Sat, Aug 7, 2021 at 12:40 AM Vaideeswaran Ganesan <vaidees@gmail.com> wrote:
Vaideeswaran Ganesan writes:
I actually want to avoid get, post, put, 2xx, 4xx codes in the client portions of the code.
I think this goal is too high-level for the standard library. I don't know what you expect on the other side, but in an application I work on it matters whether you're using GET or POST, whether you use POST, PUT, or PATCH in the application's REST API. I don't see using something other than those identifiers in the API, because they have well-defined semantics. If OpenAPI uses different identifiers, I would very likely avoid using it. The (apparent) mapping of AttributeError to "not 200" is pretty questionable, too. I could see providing an IntEnum for the standard status codes, but avoiding them entirely and instead catching a Python Error that doesn't seem to have an obvious mapping to the API's statuses doesn't seem like a great idea. If you conceal the status codes, what is the client supposed to do with an application-specific code, or a new code added to the standard?
(I don't think it's pythonic).
Not clear what you mean by that. Saying what you mean in the established language for those semantics seems Pythonic enough to me.
Not gonna fix the garbage text/plain from your mail client, but even so
looks more useful than
to me. Also, the two examples seem to have very different semantics. The first not only POSTs the items but also presents some sort of auxiliary data (the X-Token). The second appears to be implicitly a fetch despite the function name, but whether it's a GET or a POST with implicitly specified content, and what's in "creds" is quite unclear; I guess that's probably the X-Token again, but whether that is going to be the X-Token HTTP header or part of the POST content is anybody's guess. If you want to make a convincing argument you're going to have to provide better examples and more discussion of them.
It is converting the REST API syntax into language constructs. If such mapping doesn't exist - I agree with you that we probably should give flexibility to user. On the other hand, if such mapping exists and I strongly believe so - seeing several examples in web, we should allow it. Intent is to make a web interaction as if you're working with local function. This is very well known design pattern called Proxy. It also simplifies the programmer allowing same experience whether you are working with local or remote interfaces. I know we are used to one paradigm. But sometimes, we need to see if others also are applicable in relevant cases. Given the examples I have seen (swagger, sushi), we should see where this could be relevant. On Sat, Aug 7, 2021, 13:14 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Vaideeswaran Ganesan writes:
Intent is to make a web interaction as if you're working with local function. This is very well known design pattern called Proxy.
Sure, but the stdlib is fairly low-level. The question is not whether this module is useful. I'm sure for some purposes it is very useful. The question is is it useful for enough common Python tasks, or is it so likely to be frequently implemented incorrectly, that we should provide *and maintain indefinitely* this module. This is a big problem for this particular module. "Maintenance" often is more than bug-fixing, it includes developing new features. I'm pretty sure that this module would attract a very large number of requests for enhancement, often contradictory (eg, "implicit fetches should use GET, vs. implicit fetches should use POST"). I don't think that's a good use of core resources as the proposal has been presented so far.
It also simplifies the programmer allowing same experience whether you are working with local or remote interfaces.
Yes. Please don't assume people who are unfavorable to your proposal are unaware of its theoretical advantages. Show us parallel code examples with at least very close semantics. If you won't/can't do that much, many of us will make very unfavorable assumptions about your willingness/ability to maintain the module (and somebody needs to promise to do that, with you the obvious candidate).
Given the examples I have seen (swagger, sushi), we should see where this could be relevant.
That logic is invalid for Python. What we *should* do is make sure that Python works as documented (the fundamental contract of any software) and provides the *necessary* tools to get work done (the Python-specific "batteries included" policy). We provide the hammer and saw, which are not so easy to make in your home workshop. You use those tools to build a workbench. As useful as a workbench is, Python doesn't need to provide it once it does provide the hammer and saw. Obviously, this is a matter of judgment. Is pyopenapi more like a hammer or a workbench? At first glance, I say it's like a workbench. It's your job to convince this list (you don't need to convince me personally), and eventually enough core committers, that it's enough like a hammer to be included. As for swagger and sushi, I haven't seen them, and I don't intend to look at them because I have more than enough frameworks that I'm already using that I'm dangerously underinformed about. Where I have time, I study them, I don't look for more trouble elsewhere. So presenting information about examples is *your job as proponent*. The *default* is to continue the status quo (you can freely contribute the module on PyPI, which is not at all a bad deal). *You* need to show that this module benefits *Python* if included; Python needs no reason at all to say "put it on PyPI, please, so if I ever want it I can find it easily". Steve
Thanks Steve, This is very useful and actionable feedback. I'll work on the parallel examples. I'm pretty much sure, it might take few rounds of consultation to bring out the value. I will also seriously think through the PyPI as an approach, as I seem to feel that OpenAPI is not really "all that universal". Regards, Vaidees. On Sun, Aug 8, 2021 at 8:34 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
participants (4)
-
Damian Shaw -
Gustavo Carneiro -
Stephen J. Turnbull -
Vaideeswaran Ganesan