There's a chance that the algorithm used by the quiver callback (used by annotate_velocity) is not doing the correct thing for non-Cartesian data near the origin.
Is there any chance you can make a script that demonstrates the issue you're seeing? If none of the public test datasets at yt-project.org/data
are suitable for reproducing this issue, it would also help if you can make a test dataset available so that we can reproduce the behavior you're seeing. You can use the yt curldrop to share data: