Update: more workbenches seemed to do the trick. I'm currently building a stone tower over the area, and no more annoying interruptions! Thanks everyone for the suggestions