The wood shield will block his attacks, both the lightning and the antlers. Keep the shield up until he uses antlers, then you can get a couple hits in before he runs around.

But yeah, bosses don't despawn if you don't kill them, and it's also possible to summon more than one at the same time, so don't spam the offerings.