There is no way to despawn/delete the caves/burial chambers.

Best bets: move onto a new island, build over them, or create a new world.

Or like many others you could design a build using them as highlights in your camp such as placing a portal or workbench just in front of the entrance to them.