Hmm. This has never happened to me; I've always been able to swim back just fine.

Maybe there was one of those roots in the way, and you somehow swam past it when entering the passage?