Put down some no spawn items. Makes a 20m buffer.
I always dig down then place a little roofed bench plus portal and chest.
Throw a couple torches up around the edges. Throw stone and ore in chest, ferry stone through the portal for later use. Run silver up/down the mountain when I'm done with the vein.