We’ve been launching nuclear reactors into space for decades (mostly RTGs) they’re just much smaller. There isnt any chance of them exploding or anything when exposed to radiation, but yes the chance of the rocket failing, exploding and showering radioactive material over the ocean is why this has to be done incredibly carefully if it is done.