Depending on your hardware, smoke and fire slow things down a lot more than just a big house. I run around my friend's huge castle just fine, with thousands of build pieces. But put just 6 fireplaces next to one another all lit at once...hoo boy.
But it's true it's gotten better with the last update.