Yea, they are annoying as hell.
You are trying to sleep - they are near.
You are mining ore - they are near.
You are building your base? Get a rock in your face.
You have a base in the middle of water, but still in the meadows? They spawn in the water, swim to your bace and start to throw rocks (WHAT?! how?!) You start to shoot arrows at these bastards, but then a bug occures and one of them becomes untargetable - you cannnot hit him. So you run over to the shore and wait for him to come to you.
I hope they will buff fire`s repeling range or make that if you have more than ~30 armor, they run away from you.