It's a pretty minor issue, but it can be fixed with a mod like "Interact While Building" by tonsit, just search it up on nexusmods.