Definitely a lag issue. A buddy was hosting for me and two others and everyone was floating, falling through the earth, general lag was reported from everyone (his wifi is known to be ass). Then i hosted a world and everyone said all that stopped, even my bud that has ♥♥♥♥ wifi.

You could try to have your friend host a world and see if its any better for all parties. If not, then not much you can do.