From what I understand, it is not currently possible to do this. The inventory is currently player based, not server based. Reason for this is that if you remove someones item now, they also loose it everywhere else too. As much as I understand your problem, it would be a bigger problem if someone invites players just to troll with them by removing all their items.