When you aren't around the area doesn't exist in a dynamic sense. It's paused and nothing can happen, it doesn't exist in game without a player nearby. So those things are only happening when you are near them. They can load in fairly far, 60m+ at least so you need to a) have walls and such and b)inside those walls have no spawn items so enemies can't spawn inside while you approach.
( https://valheim.fandom.com/wiki/Player_base )
That Ward protects nothing from damage. It prevents other players from opening stuff or building/deconstructing via hammer, and chimes with a blue aura whenever anything in its radius is attacked/damaged.