You could always just type in devcommands, debugmode and fly back. Also, you most likely are entering to the portal too soon after switching the tag, this has happened to me.