On the contrary, on sets you can apply union *and* difference. And since union seems the exact contrary of difference, it's illogical that | is used instead of +.
Set union is self-evidently NOT the inverse operation of set difference. It's not merely not an obvious analogy, it's simply wrong.
If we had a multi-set type (collections.Counter is close, but not exact), then there would be something close to an inverse of set difference. But that's not the type Python provides (nor should it as a built-in).
Set union, however, has a great deal in common with bitwise-or, so using the same symbol is intuitive.