Of course they can underflow. But I don't think that's a practical problem except in very rare cases. It means you're explicitly asking for better than +/- 2**min_exp, so it shouldn't be surprising that nothing but an exact match qualifies.
Take a concrete example: with a tol of
1e-5, it's only going to underflow if expected is around 1e-320 or below. But since the next smaller and larger numbers (9.95e-321 and 1.0005e-320) aren't within 1e-5, the test gives the right answer despite underflowing.
I'd have to think about it a bit to make sure there's no pathological case that doesn't work out that way--but really, if you're checking subnormal numbers for closeness with a general-purpose function, or checking for relative closeness pushing the bounds of 1 ulp without thinking about what that means, I suspect you're already doing something wrong at a much higher level.
So, just special-casing 0 should be sufficient.
Maybe the answer there is to have an is_close_to_0 function, instead of a parameter that's only useful if expected is 0? But then you might have, say, a comprehension where some of the expected values are 0 and some aren't, so maybe not...