Yeah they can get annoying but they are easy to kill. Its just something you gotta get used to. But I wouldn't remove them because they are meant to be an easier enemy to start out with early game. Also, their drops are quite useful especially if you are building. You usually don't have to deal with them if its day so just sleep when its night.