Sure, break it and rebuild it since you get all the materials back. You'll need to use an axe or something, not the hammer.

Otherwise, sometimes I just jump out and push it while on land or in the water.