Placing workbenches will deter spawns, so your best bet is to clear the island and start putting up some workbenches in the area you're working on. Then slap some walls around that sucker and you should be golden.

No, you cannot destroy crypts or caves. You can dig under them to make mini cave bases if you wish.