Trolls get shredded by any arrows, even wooden arrows. But their gigantic AoE attacks are very humbling for a first time builder.

Start with a fence. Always cover your entire land in a fence, water access also helps to protect one side. The fence will force trolls out of range so they can't crush dozens of bricks with a single swing. A decent size fence may be 300 or so wood, but the square cube law means that a slightly longer fence will cover far more land area.