Probably waves from water sometimes coming up too high and temporarily putting the wood underwater. Thus damaging it.

Nothing you can do about clipping water waves through terrain, sadly. Other than just building higher...