Yes they can be attacked. I am lazy and only sometimes house mine. Many moons ago I was running from a troll and jumped thru a portal. By time I arrived on far size the portal back no longer worked because the troll had smashed it behind me.