The devs have made it pretty clear that it's intended to work this way. Is it dumb? Yeah, very much so, but it's not a bug.

Thankfully there's a mod for this: https://valheim.thunderstore.io/package/Azumatt/ImFRIENDLY_DAMMIT/