All building games are this way, only so much can be built in an area before effecting performance.
With normal games, all the background stuff you see, all the buildings, all the landscape, everything you can't really alter, has been combined into one big object. This means its only one item the game has to keep track of. Then each thing that can be altered, be it an item moved or a door opened or closed, or something that can be blown up, are then separate items.
In a game like Valheim, because everything can be altered and changed at any time, all of it has to be tracked as individual pieces. So every single pole, every single fire, every single decoration you place has to be tracked separately. It can't be combined. So instead of tracking dozens or hundreds of active objects, it now has to track thousands or tens of thousands of different objects, these are you "active instances".
So the performance impact isn't about Graphics, but about the processing needed to actively track all those items in your area.
You can use F2 to view your active instance count. Around 15k is when some people start having performance impact, lower/higher depending on the individual system.
Optimization of the game will only improve this so much, there will always be limits as the game has to track ever single item that is actively loaded.
Note that higher overhead demand items will cause more of a performance hit, such as dynamic light sources and those that can generate smoke.
Lots of little buildings nearby won't be better than a large one if they are in the same area, as they are still all active instances at the same time.
You can however build different bases a distance from each other so they aren't loaded in at the same time just fine.
Note terrain alterations have to be kept track of as well and those up your active instance count in an area as well, just like building. Basically any changes from the default state of the world.
Just keep an eye on your FPS and your active instance count to get an idea of what the upper limit is for your system.

Originally posted by Esos:

- but why this game so laggy when having a huge building or having a lot of stuff nearby even when running in Vulkan. -

Originally posted by Weaver:

- So every single pole, every single fire, every single decoration you place has to be tracked separately. It can't be combined. So instead of tracking dozens or hundreds of active objects, it now has to track thousands or tens of thousands of different objects, these are you "active instances".
---
Lots of little buildings nearby won't be better than a large one if they are in the same area, as they are still all active instances at the same time. -

Even if your buildings are loaded at the same time, having them smaller and disconnected should, at least in theory, improve performance. The game is constantly checking if each individual piece of the building is stable. Smaller and disconnected buildings make those calculations easier.

Systems with 3090's have the same issues as well. Dont worry its not your system and dont let anyone tell you it is.

The game right now hasn't had any big optimizations yet and wont for a little while. So build small and thin for now if you want more frames. Like 1 house with all the crafting things you need and make another building a bit far away for anything else. Spread your stuff apart for the best results.

Originally posted by PsychaChi:

Systems with 3090's have the same issues as well. Dont worry its not your system and dont let anyone tell you it is.

The game right now hasn't had any big optimizations yet and wont for a little while. So build small and thin for now if you want more frames. Like 1 house with all the crafting things you need and make another building a bit far away for anything else. Spread your stuff apart for the best results.

Glad to hear that, thanks!
That's a great idea to build a small building separated from one another, I'll try this later.