If it's any help, I would suggest looking at how SymPy does its PDF documentation. We have a few adjustments to the Sphinx defaults to make things work (although it's honestly not that much, mainly just using XeTeX for Unicode support).
https://github.com/sympy/sympy/blob/master/doc/Makefile
I personally don't use PDF documentation but I know for SymPy, many people do, which is why we have continued to support it.
Also, one final thing I will note is that if you have any LaTeX math in your documentation, the PDF build is the only way you can be sure that that LaTeX is valid. MathJax only renders the LaTeX in the browser when the page is loaded so you will only notice invalid math once you look at it and see it gives an error. To contrast, the PDF docs fail at the build stage if the LaTeX is invalid. Obviously we have a lot of math in the SymPy docs but I don't know if there is any in the NumPy or SciPy docs. This does also mean that to be useful you actually need to build it on CI. If you don't, you will end up with docs that don't actually build any more by the release because of unseen errors (even without LaTeX math this will likely end up happening, so I would recommend against this).
Aaron Meurer