I like to mix a boar ranch and bee hives with my farms and surround them with my tree farms. Once every four or five days I show up, immediately cull and feed the boars, then spend the entire day there tending the crops, cutting wood, and ofc harvesting honey.

If you arrange things as an isosceles triangle with boar farm at center, you can have three workbenches/portals/whatever without ever coming into 40 meter range of more than two base structures at a tie - so no raids. Also in the meadows so don't need more than a pole fence to keep wildlife at bay.

The piggies, like the crops, will get matured according to time elapsed on your next visit. Great way to spend rainy days or to just take a break from your normal activities.

[Edit:] fixed typos and layout description. Also direct answer to OP question: yes, they'll grow.

Things that won't happen when you are far away: Animals won't reproduce, lifeforms won't attack tamed animals, other lifeforms or structures

Far away zones go into suspended mode. Nothing will move and hostiles won't spawn.
When you return, crops will wake up and rapid age to their proper state.

Crops will grow. Your lifestock will survive. You don't need to worry about them being attacked, as long as you are not there.

Same with the kiln and the smelter. You can fill them up and go somewhere else and they will continue to process as long as you are in game. If you are playing multiplayer on a dedicated server, as long as there is one player on the server time will pass, crops will grow, wood will become coal and ore will smelt.