What do youe evenb mean?

adding them to permittedlist.txt ? I'd suggest reading the server logs when one xboxer tries to join and compare it to what happens when you join.

Missing -crossplay on the servers command line?

Or do you mean make them admins on your server? -> adminlist.txt? see above...