If you're going to be in the area for awhile, dig a hole and push the greys into it. They don't respawn if they don't die. But they will continue cursing and throwing rocks at you from their pit.