It seems to me that moving PyObject* to be a handle leaves you in a
place very similar to HPy. So why not just focus on making HPy suitable
for developing C extensions, leave the existing C API alone, and
eventually abandon the existing C API?
I agree (but I'm biased :)), but I think there is also an important point which is easy to miss/overlook: it is not enough to declare that now you have handles instead of refcounting, you also need a way to enforce/check that the handles are used correctly.
CPython might declare that object references are now handles and that each handle must be closed individually: this would work formally, but as long as handles are internally implemented on top of refcounting, things like closing the same handle twice would just continue to work if by chance the total refcount is still correct. This means that we will have extensions which will be formally incorrect but will work well on CPython, and horribly break as soon as you try to load them on e.g. PyPy.
That's the biggest selling point of the HPy debug mode: in debug mode, HPy actively check that handles are closed properly, and it warns you if you close a handle twice or forget to close a handle, even on CPython.