Yes, you have to prepare before lumberjacking: rested, fed, and wearing upgraded "blue" armor. They can't one-shot you this way. But they still can two-shot you.