I had that problem with a simple pole fence. I think mobs or deer wander by and aggro the pigs. The pigs go nuts to get to them and break down the fence. Once I put up a bigger, outer wall to keep all the mobs away, the piggies stay home and leave the fence alone.