"doesn't the totem solve this issue?"

I take it that you mean the "ward" - no, that does not prevent the destruction of things; it only prevents other players from opening "your" doors and chests and also gives a warning tone when someone attacks your stuff - e.g., walls and chests.