I dunno, it's almost like the player characters themselves not being natural inhabitants of the world could enable only them and their possessions (aside from processable metals) to operate portals. What do you think about that?