Moats. Dig moats around your base, then build a bridge across, but don't connect it. This way you can simply jump back and forth and they can't get in.

Also build your wall two high and they can't hit you with rocks. That or simply dig the moat wide enough they can't throw them far enough to hit you.