You could, but you would be missing out on some of the hoe's best potential by not using it. I for one typically use the hoe to make large winding paths in the swamp to better help navigate or use a cart if need be. Helps keep the leeches at bay too.

Also can't stand not having a flat area to work with when building a new dwelling so theres that too. Oh & using earth pillars for support helps a bunch for castles/ large builds too.