They seldom do, but yeah, enemies do attack portals when they come close.

Normally if you're not there, you can leave them in the open and just repair them once in awhile. But when there are raids or troll attacks, it's best you lure them as far away as you can.