On the topic of NaNs: I think in the face of ambiguity, we should refuse the temptation to guess and raise ValueError when the values given are not comparable to the point of clearly determining whether `value` is or isn't within `minimum` and `maximum` or which of the two bounds it exceeds. It is the programmer's responsibility to ensure they supply arguments that make sense, and non-numbers should be considered exceptional behaviour: as such, the programmer that anticipates those kinds of values should handle them exceptionally.
About the stdlib/builtin status: `min(max(x, min_), max_)` is a common enough idiom for a simple enough function that I think it is definitely beneficial and in no way dangerous to have, and that any status other than builtin would incur too much overhead for something so small. I risk claiming that most calls to max() and min() occur in modules that do not `import math` (or at the very least, they occur often enough without `import math`), and I believe clamp(), being very similar in scope to those, should benefit from similar status.
I don't think clamp is a common enough identifier in non-specific fields to disqualify its implementation as a builtin by itself. By contrast, terms like `min` and `max` are extremely common in pretty much all contexts, and yet we've all survived just fine by typing `min_` and `max_` or even shadowing the names altogether (which I prefer not to do, but is definitely an option).