It's more of a metaphor. He's telling you it suits your personality.

Basically, he thinks you're akin to worms