I almost never chop down big trees unless they're too much in the way, apart from birch and pine. I don't even like felling oaks.

I clear out the small trees, as well as any logs or stumps, and destroy ruined buildings that I find. Even entire villages. Never had trouble with regular wood even though I don't touch beech or fir trees unless necessary. A portal works great for the building demolition, as I can just return to base to drop off wood and then return to destroy and collect more.