If you dig a pit that is like, 1 wall deep or better, with steep sides, they cannot get to you. Just trench in the mine, finish mining, then defeat the golem after (you know, so you can leave and stuff).

If you face tank them, you'll want black metal shield and padded armor to reduce stagger. Some folks swear by rolling dodge + mining pickaxe.. but I find wooden arrows from a pit do just fine, or any other weapon if you meat shield them. Takes a bit. Be prepared for nightfall, and adds respawning.