Originally posted by Danfeop:

you are within render distance so boat and enemies stay rendered. If youre sitting near it make sure it it safe or break it and stash materials in a chest. Otherwise just move away from it so enemies to spawn there.

I was at the other island so I'm sure it is not rendering issue. Plus then the materials should on the ground. There were no materials in sight whenever I came back.