ast.parse, ast.dump, but with comment preservation?
samue...@gmail.com
samuelmarks at gmail.com
Tue Jan 11 00:30:39 EST 2022
On Thursday, December 16, 2021 at 5:56:51 AM UTC-5, lucas wrote:
> Hi !
>
> Maybe RedBaron may help you ?
>
> https://github.com/PyCQA/redbaron
>
> IIRC, it aims to conserve the exact same representation of the source
> code, including comments and empty lines.
>
> --lucas
> On 16/12/2021 04:37, samue... at gmail.com wrote:
> > I wrote a little open-source tool to expose internal constructs in OpenAPI. Along the way, I added related functionality to:
> > - Generate/update a function prototype to/from a class
> > - JSON schema
> > - Automatically add type annotations to all function arguments, class attributes, declarations, and assignments
> >
> > alongside a bunch of other features. All implemented using just the builtin modules (plus astor on Python < 3.9; and optionally black).
> >
> > Now I'm almost at the point where I can run it—without issue—against, e.g., the entire TensorFlow codebase. Unfortunately this is causing huge `diff`s because the comments aren't preserved (and there are some whitespace issues… but I should be able to resolve the latter).
> >
> > Is the only viable solution available to rewrite around redbaron | libcst? - I don't need to parse the comments just dump them out unedited whence they're found…
> >
> > Thanks for any suggestions
> >
> > PS: Library is https://github.com/SamuelMarks/cdd-python (might relicense with CC0… anyway too early for others to use; wait for the 0.1.0 release ;])
Ended up writing my own CST and added it to that library of mine (link above).
My target is adding/removing/changing of: docstrings, function return types, function arguments, and Assign/AnnAssign. All but the last are now implemented.
I was careful not to replace code elsewhere in my codebase, so everything except my new CST code (in its own files) stays, and everything else works exclusively with the builtin `ast` module as before.
More information about the Python-list
mailing list