Are you playing with others on a server? If so you could have been burgled. Or another person was near there and collateral damage. They might not have noticed or been embarrassed and hoped you wouldn't notice.
If solo, nothing should happen when you aren't there. If you are solo and somewhere else on that seed then stuff will keep growing. If you log off then everything stops, time does not march on. If you are on a server with others then time clicks by as long as someone is on.
One thing I always do is keep a few of every seed or plant somewhere else safe in case everything gets destroyed. Which was good the time I didn't realize I planted the only other barley I had in meadows. Derp.