yeah wait till you are in the swamp and have obtained some iron. aside from access to stone blocks that will also give you more efficient tools that make farming much less of a hassle especially if you are going for a building project like this