If you don't allow for PvP, they cannot harm you. It is still advised to rather join them "fresh" and naked.

Mind, they will still be able to loot your corpse, if you die and they happen to get to it first. At least as far as I know.

Oh, and they will totally be able to wreck your base and steal everything from you once you get situated.