True magic is not something to weild without consequences. Mugin says Jotunn were ousted which implies they were forcibly taken out but maybe that word play could make you think that but could literally mean they were ousted by the world tree for being unresponsible with their magic use