I'm not against renaming a wheel to adjust the tags, but it is a strange caching strategy. wheel is due for a 'retag' feature that does a more rigorous job of changing tags.
The build system should name its wheels correctly. Normally the pyNN tag would be used for a pure distribution and the cpNN tag when there are extensions. If you want to affect the tagging when called through PEP 517, edit the source's pyproject.toml / setup.cfg / setup.py.
It will probably always make sense to use pip to build wheels when you want to build wheels for a project and all its dependencies. Then you can keep these wheels in a folder for repeatable offline deployment.