You must stay within the raid radius for it to time out.

If you keep spawning in a bad place just destroy your bed and you will respawn in the stone circle you entered the game.

I think if you wait for a new raid it will remove the old one. So it can end that way as well.

Lesson here is to never make a bed a spawn point without a portal.